Chain-of-Thought › Self-Consistency

A natural extension of chain-of-thought prompting is to generate not one reasoning chain, but many, and then aggregate their conclusions. Self-consistency (Wang et al., 2022) does exactly this. The model samples multiple diverse reasoning chains for the same question, and the final answer is determined by majority vote over the conclusions of those chains.

The intuition is that there are many valid ways to reason toward a correct answer, but fewer ways to reason toward any specific incorrect answer. Errors in reasoning chains tend to be idiosyncratic, arising from specific wrong assumptions, arithmetic mistakes, or misreadings that differ from chain to chain. The correct answer, by contrast, tends to be reached by multiple independent chains that decompose the problem differently but converge on the same conclusion. Majority vote over conclusions therefore favors the correct answer.

This framing makes a probabilistic prediction: the correct answer should be the mode of the distribution over sampled answers, and this mode should become more reliable as the number of samples increases. Empirically, this prediction holds well. The improvement from self-consistency is consistent across model sizes, tasks, and benchmarks, and continues to grow with sample count, though with diminishing returns.

Self-consistency substantially improves accuracy over single-chain CoT, particularly on mathematical and symbolic reasoning tasks. On GSM8K, self-consistency with 40 sampled chains improves accuracy by 10-20 percentage points over single-chain CoT with the same model. The improvement is most pronounced for problems where CoT single-chain accuracy is in the 50-70% range, exactly the range where errors are common but not universal.

For self-consistency to work, the sampled chains must be diverse. If you sample with temperature 0 (greedy decoding), every chain is identical and you gain nothing. You need to use a nonzero temperature to introduce variation across samples. Wang et al. (2022) found that temperatures between 0.5 and 0.8 work well: high enough to produce distinct reasoning paths but not so high that chains become incoherent.

The diversity of chains matters for correctness voting and for the richness of reasoning strategies explored. At a moderate temperature, different sampled chains may approach the same problem via different decompositions, catching errors that any single decomposition might miss. One chain might compute earnings per day first, then per week, then per three weeks. Another might compute total hours worked first, then apply the hourly rate. Both arrive at the same answer via different arithmetic paths, and their agreement provides stronger evidence of correctness.

The cost is computational. Sampling 40 reasoning chains multiplies inference cost by 40. This trade-off is worthwhile when accuracy is critical and latency is flexible, but impractical for real-time applications. Most production deployments that use self-consistency settle on 8-16 samples, capturing most of the accuracy improvement while keeping costs manageable. The self-consistency accuracy curve shows strong diminishing returns beyond 16 samples, so few applications justify the full 40-sample budget.

Universal self-consistency (Chen et al., 2023) proposed a more efficient variant: rather than running full generation for every chain, use the first-pass reasoning chains to identify low-confidence questions (those with high variance in sampled answers) and apply multi-sampling only to those. Questions with high first-pass confidence do not benefit much from additional samples, so the budget can be concentrated where it helps most.