Reasoning Verification › Step Verification

Step verification is the practice of checking each individual step in a chain of thought for correctness, rather than waiting until the end and evaluating only the final answer. It treats a reasoning chain as a sequence of claims, where each claim either follows correctly from prior steps or it does not.

The motivation for step-level checking is straightforward. If you can catch an error at step 3 of a ten-step problem, you can discard that reasoning chain immediately rather than allowing nine additional steps of wasted computation. More importantly, you can diagnose what went wrong. An outcome-level checker tells you that the answer is wrong. A step-level checker tells you where the reasoning broke down, which is far more actionable.

The first question is how to define a step. In natural language reasoning, step boundaries are not always obvious. A step might be:

• A single sentence in a chain of thought ("Since 3 is prime, we know...")
• A logical inference ("Therefore, by the triangle inequality...")
• A single arithmetic operation ("12 times 7 equals 84")
• A complete intermediate conclusion in a multi-part problem

For mathematical reasoning, steps often correspond to individual equations or transformations. For code generation, a step might be a single function or a block of logic. For multi-hop question answering, a step is typically a single sub-question answer that feeds into the next.

The key property a step must have is that it should be checkable in isolation given the prior context. A step like "therefore x = 5" can be verified by checking whether the preceding equations imply this. A step like "this is probably correct" cannot be verified because it makes no specific checkable claim.

In practice, different researchers have used different granularities for step definition, and the choice matters for both annotation and model performance. Finer-grained steps (individual arithmetic operations) enable more precise error localization but require more annotations and more scoring passes. Coarser steps (complete sub-proofs) are easier to annotate but may miss errors within a step. The right granularity depends on the domain and the kinds of errors that commonly occur.

Understanding what can go wrong in a reasoning step helps clarify what verification must catch:

• Arithmetic errors: A calculation is performed incorrectly ("12 times 7 is 82" instead of 84). These are among the most common errors in mathematical reasoning and are often not detected by fluency-based evaluation.
• Logical fallacies: A conclusion does not follow from the premises even if each premise is individually correct. For example, concluding that "if all cats are animals and all animals breathe, then all breathing things are cats" confuses a valid syllogism.
• Unjustified assumptions: A step introduces a new claim that was not established by prior steps and is not obviously true. This often manifests as "clearly, since X..." when X has not been established.
• Scope errors: A step applies a rule outside its valid domain, for example applying a property that only holds for positive integers to a negative number, or applying a continuous approximation where discrete counting is required.
• Factual errors: A step asserts a fact about the world that is incorrect, such as misremembering a physical constant, a historical date, or a mathematical identity.
• Reference errors: A step incorrectly cites or misinterprets a result from earlier in the chain, carrying forward a value that was computed differently.

A step verifier must catch all of these. This is a challenging requirement because it demands both mathematical rigor and world knowledge, and because some error types (unjustified assumptions, scope errors) require understanding the domain context in which the step is embedded.

Step verification can produce either a binary signal (this step is correct or incorrect) or a graded signal (this step has probability $p$ of being correct). Binary labels are simpler to collect and use but lose information. Graded signals preserve more information about uncertainty.

In practice, most systems use a probability score. A step verifier assigns a score in the range $[0, 1]$ to each step, where 1 means certainly correct and 0 means certainly incorrect. This score can then be used to select among multiple candidate reasoning paths, weight different solutions, or flag uncertain steps for human review.

The graded formulation also handles ambiguity better. Some steps are correct but expressed ambiguously. Some are incorrect but in a minor way that does not affect the final answer. A binary label forces a choice that a graded probability can express more faithfully. When using PRM scores to guide search, the continuous score allows ranking partial chains by quality, which is more informative than simply filtering by a binary threshold.

To make this concrete, consider a simple arithmetic word problem: "A train travels at 60 mph for 2 hours, then at 80 mph for 3 hours. What is the average speed for the whole journey?"

A model might produce this reasoning:

Step 1: Distance in first segment: 60 times 2 equals 120 miles. (correct) Step 2: Distance in second segment: 80 times 3 equals 240 miles. (correct) Step 3: Total distance: 120 plus 240 equals 360 miles. (correct) Step 4: Total time: 2 plus 3 equals 5 hours. (correct) Step 5: Average speed is (60 plus 80) divided by 2 equals 70 mph. (incorrect: uses arithmetic mean of speeds instead of total distance over total time)

The correct answer is 360 divided by 5 equals 72 mph. A step verifier that checks step 5 can catch this specific error: the step applies the formula for arithmetic mean speed rather than the definition of average speed. An outcome verifier might catch that the final answer is wrong, but cannot pinpoint where the error entered.

This kind of error, substituting a convenient but incorrect formula for the right one, is extremely common in language model outputs. Step verification provides the machinery to identify these cases reliably.