Reasoning Strategies › Self-Consistency

The simplest chain of thought prompting generates one reasoning path and returns its answer. This is brittle. A single path can go wrong at any step, and there is no mechanism to detect or correct errors. Self-consistency fixes this by sampling many reasoning paths and selecting the most common answer.

The core intuition is straightforward: if you ask a human to solve a math problem and they get the same answer by three different methods, you should trust that answer more than an answer reached by only one method. Self-consistency applies this logic at scale. It does not require the model to become more accurate on any individual reasoning attempt; it requires only that correct answers cluster while errors scatter.

Self-consistency, introduced by Wang et al. (2022), replaces greedy decoding with a sample-then-aggregate approach. The procedure is:

1. Given a problem, sample $N$ diverse reasoning paths from the language model using temperature sampling (typically $T = 0.7$)
2. Extract the final answer from each reasoning path
3. Return the answer that appears most frequently across all $N$ paths

The key insight is that diverse reasoning paths tend to agree on correct answers and disagree on incorrect ones. Errors are idiosyncratic: one path might make an arithmetic mistake, another might misinterpret a condition, but correct answers tend to converge. Each error has its own unique signature. The correct answer, by contrast, has only one form: it is whatever the problem demands.

Formally, if $f(\mathbf{x})$ denotes the language model and $a_i$ is the answer extracted from the $i$-th sampled path, the self-consistent answer is:

$$a^* = \arg\max_{a} \sum_{i=1}^{N} \mathbf{1}[a_i = a]$$

where:

• $a^*$: the final answer returned by self-consistency
• $N$: the number of sampled reasoning paths
• $a_i$: the answer extracted from the $i$-th reasoning path
• $\mathbf{1}[a_i = a]$: an indicator function equal to 1 when the $i$-th answer matches candidate $a$

This is majority voting over answers, not over full reasoning paths. Two paths that reach the same answer by completely different routes both contribute to that answer's vote count. This is an important design choice: voting over full paths would require comparing free-form text, which is fragile and slow. Voting over extracted answers is simple and robust, provided the answers can be normalized into a canonical form.

The effectiveness of self-consistency rests on a probabilistic argument. Assume that the language model assigns higher probability to correct reasoning steps than to incorrect ones, but that this probability is not 1. Then a single path will occasionally err. Over $N$ paths, errors are distributed randomly across many possible wrong answers, while correct answers concentrate in one place.

The voting step exploits this asymmetry. If the correct answer $a^*$ appears in fraction $p$ of paths, and all wrong answers are distinct, the correct answer wins as long as $p > 1/k$ where $k$ is the number of distinct wrong answers. In practice, $k$ is large and diverse, so even modest $p$ (say 0.4) suffices for the correct answer to win the plurality.

This argument illustrates why the diversity of wrong answers matters as much as the accuracy of individual paths. If the model tends to make the same systematic error over and over, wrong answers will cluster just as strongly as correct ones, and voting will not help. The strategy works because language model errors tend to be distributed across many different wrong values, not concentrated on one wrong answer.

To see why errors scatter, consider what happens when a model makes a mistake in a multi-step arithmetic problem. In step 2 of 5, the model might misread "twice as many" as "half as many." The downstream arithmetic in steps 3 through 5 will then produce a wrong final answer, but one that is arithmetically consistent with the mistaken premise. A different error, misreading "5 fewer" as "5 more" in the same step 2, produces a different wrong answer through a different downstream chain. These two errors generate numerically distinct wrong answers that split the vote, while the correct reasoning path concentrates votes on the true answer.

Empirically, Wang et al. showed dramatic gains. On the GSM8K benchmark, chain of thought prompting with a 540B parameter model achieved 56.5% accuracy. Self-consistency with 40 samples raised this to 74.4%, an 18-point improvement, without any additional training. The gains were even larger on harder benchmarks, suggesting that the harder the problem, the more individual paths err, and the more aggregation helps.

Self-consistency trades compute for accuracy. Sampling 40 reasoning paths costs 40 times as much as sampling one, which is prohibitive in latency-sensitive applications. For a model that takes 2 seconds to generate a reasoning path, self-consistency with 40 samples takes 80 seconds, not 2. This is acceptable in asynchronous batch settings but unacceptable in interactive applications.

Several practical considerations govern the deployment of self-consistency:

• Number of samples: Gains plateau around $N = 20$ to $40$ for most tasks. Sampling more than 40 paths yields diminishing returns because the aggregate vote has already converged.
• Temperature: Use $T = 0.7$ to $T = 1.0$ for diversity. Greedy decoding ($T = 0$) produces nearly identical paths and defeats the purpose. If all paths are the same, you get zero benefit from voting.
• Answer extraction: The aggregation step requires extracting structured answers from free-form text. For math problems, this usually means finding numbers; for multiple-choice problems, it means finding a letter. Inconsistent extraction errors propagate directly to accuracy, so robust extraction is critical.
• When to use it: Self-consistency helps most on tasks with verifiable answers (math, logic, coding) where the model's reasoning is uncertain. It helps less on open-ended generation tasks where answers are not comparable, because there is no well-defined way to vote over free-form text.

One refinement is weighted self-consistency, where each reasoning path is weighted by the model's log-probability rather than counted equally. Paths the model is more confident about contribute more to the final vote. This can help when some paths are clearly stronger than others, but the improvement over uniform voting is often modest in practice.

Understanding when self-consistency breaks down is as important as understanding when it works. The strategy can fail in three ways.

First, if the model has a strong systematic bias toward a particular wrong answer, errors will concentrate rather than scatter. This can happen when a problem has a common confounding factor: a unit conversion error that most paths make, or a conditional that most paths misread. In these cases, the wrong answer wins the vote because it appears in more than half the paths. Self-consistency amplifies the model's biases rather than correcting them.

Second, if the problem requires rare or specialized knowledge that the model barely has, nearly all paths will produce wrong answers from different corners of the wrong-answer space. The correct answer may appear in very few paths, and some wrong answer will win by plurality. More samples help, but if the underlying per-path accuracy is very low (say, 10%), you would need hundreds of samples for reliable majority voting.

Third, answer extraction can become a bottleneck. For problems where the final answer is embedded in long reasoning text, inconsistent extraction can corrupt the vote. Two paths that arrive at the same numerical answer but express it differently ("\115$" vs "115 dollars") might count as different answers. Preprocessing to normalize answer formats is critical for reliable self-consistency.