Self-correction helps only when the model can verify

"Check your work" is among the most common instructions given to language models, but its effectiveness is not well established. We conducted a controlled study to determine when a second pass improves an answer.

The central finding is that self-correction improves accuracy primarily on tasks for which the model can reliably distinguish correct from incorrect reasoning.

Experimental setup

We evaluated 4,000 problems across four domains:

• Arithmetic
• Multi-step mathematics
• Code repair
• Factual recall

Each problem was answered once and then answered again following a fixed self-critique prompt. For each pair, we measured whether the answer changed, whether the change improved accuracy, and whether the model's stated confidence tracked correctness.

Results

In arithmetic and multi-step mathematics, self-correction changed 22 percent of answers and improved accuracy by 4.1 percentage points. The changes were strongly associated with the model's own step-level confidence, suggesting that the model could verify its work in these relatively constrained settings.

Factual recall produced the opposite pattern. Self-correction changed 17 percent of answers but reduced accuracy by 1.8 percentage points. Correct initial answers were overturned slightly more often than incorrect answers were repaired. Confidence was also only weakly related to correctness in this domain.

Code repair fell between the two. Accuracy increased by 1.9 percentage points, and nearly all of the improvement occurred on problems for which the model could run a failing test and use its result as a verification signal.

Interpretation

Self-correction is better understood as a form of applied verification than as a general reasoning capability. Asking a model to review its own work assumes that it possesses a sufficiently reliable signal for distinguishing a good answer from a bad one.

When that signal is provided by a test, a formal constraint, or a well-defined procedure, self-correction can be useful. When the model must generate the verification signal itself, the method is effective only in domains where its judgment is already reasonably reliable.

This is one reason we focus on explicit verification mechanisms rather than relying solely on longer self-critique prompts.