Chain-of-Thought › Summary

Chain-of-thought prompting is one of the most impactful developments in the practical use of large language models. By eliciting explicit, sequential reasoning before a final answer, CoT substantially improves model performance on multi-step reasoning tasks, particularly arithmetic, commonsense reasoning, and symbolic manipulation.

The core mechanism is the use of intermediate tokens as computation. Writing out a reasoning step produces context that makes the next step easier to generate correctly. This distributes complex computation across multiple generation steps rather than compressing it into a single prediction. The effective depth of computation available to the model grows from $L$ layers (the depth of the transformer) to roughly $L \times T$, where $T$ is the number of reasoning steps the chain produces.

The key techniques covered in this chapter:

• Few-shot CoT: Provide 4-8 exemplars with explicit reasoning chains before the target question. Works best with carefully constructed exemplars that demonstrate atomic, correct reasoning steps. Exemplar quality, diversity, and atomic step structure all affect performance.
• Zero-shot CoT: Append "Let's think step by step" to the prompt. No exemplars required. Surprisingly effective given its simplicity, though slightly below well-constructed few-shot CoT. The trigger phrase activates reasoning-associated contexts from pretraining.
• Self-consistency: Sample multiple reasoning chains and take a majority vote over their conclusions. Substantially improves accuracy at the cost of proportionally higher inference compute. Most of the benefit comes in the first 8-16 samples, and diminishing returns are strong beyond that.
• Least-to-most prompting: Decompose complex questions into subproblems before solving, enabling better generalization to harder problem variants by making the dependency structure explicit.
• Program-aided reasoning (PAL): Generate executable code rather than natural language chains, delegating arithmetic to a Python interpreter. Eliminates arithmetic errors and is especially effective for computation-heavy tasks.
• Tree-of-thought (ToT): Extend CoT to non-linear reasoning by exploring multiple candidate chains and using search to navigate the tree. Powerful for exploration tasks but computationally expensive.
• CoT fine-tuning: Train models directly on reasoning chain data, either from human annotation or distilled from larger models. Produces models that reason by default and partially circumvents the scale dependency of prompted CoT.
• Process reward models (PRMs): Provide step-level supervision rather than outcome-level, rewarding correct reasoning process. Most effective when combined with search over candidate chains.

CoT also has significant limitations. Reasoning chains are not always faithful representations of model computation: chains can justify biased conclusions, rationalize pattern-matched answers, or contain unfaithful explanations that diverge from the actual derivation. Performance depends heavily on model scale, with little reliable benefit below roughly 10B parameters. Hallucination can propagate through chains, amplifying early errors into large final errors. And the computational overhead of self-consistency makes some applications impractical.

Understanding both the power and the limits of chain-of-thought prompting is essential for applying it effectively. CoT is not a solution to all reasoning challenges, but used appropriately, it is one of the most reliable tools available for improving language model reasoning performance. The next chapter explores how these prompting-based reasoning techniques connect to reinforcement learning approaches that teach models to reason through training rather than through inference-time guidance.