Measuring faithfulness in chain-of-thought reasoning
A method for testing whether a model's stated reasoning reflects the computation behind its answer, applied to four open models.
When a model explains an answer step by step, the explanation often appears to describe how the answer was produced. That appearance can be misleading. A model may arrive at an answer through one process and subsequently generate a coherent explanation that merely justifies it.
If step-level reasoning is to be treated as evidence, its relationship to the model's computation must first be measured.
Method
Our method is based on a simple intervention. If a reasoning step materially contributes to the answer, corrupting that step should affect the result. If the answer remains unchanged, the step may be explanatory rather than causally relevant.
For each problem in the test set, we generated three modified versions of the model's reasoning:
- One in which a step was negated
- One in which a step was replaced with a plausible but irrelevant step
- One in which a step was removed
We then re-ran the model and recorded whether the final answer changed. We define the faithfulness score as the proportion of corrupted steps that changed the answer, averaged across the test problems.
Findings
Three findings were particularly notable.
First, scale alone produced only a modest improvement. Increasing the model size from 8 billion to 70 billion parameters resulted in an improvement of less than ten percentage points in faithfulness.
Second, irrelevant reasoning steps were the most common failure mode. They accounted for approximately half of the unfaithful steps observed across the models.
Third, adding a separate verifier that scored individual steps produced a larger improvement in faithfulness than changing the reasoning model itself. This result motivates our ongoing work on verification methods that are sufficiently efficient for production use.
Limitations
Corruption testing measures whether a step is causally relevant to the answer. It does not establish that the step is honest or that it accurately describes the model's underlying computation.
Our experiments were also limited to mathematical and symbolic tasks, where reasoning steps can be manipulated relatively cleanly. Whether the method transfers to open-ended reasoning remains an open question.