Chain-of-Thought Emergence › Chain-of-Thought Emergence

In the previous chapter, we explored how in-context learning emerges as models scale beyond certain compute thresholds. Chain-of-thought (CoT) reasoning is perhaps the most striking example of emergent capability: the ability of language models to solve complex problems by generating intermediate reasoning steps before creating a final answer. This chapter examines how CoT was discovered, the conditions under which it emerges, techniques for eliciting it, and current theories about the mechanisms underlying this remarkable behavior.

The discovery of chain-of-thought prompting fundamentally changed how we interact with large language models. Before CoT, models often failed at multi-step reasoning tasks even when they possessed the knowledge needed to solve them. A model might know that a store sells apples for $2 each and that someone has $10, yet fail to correctly answer how many apples they can buy. The key insight was that prompting models to "show their work" could make latent reasoning capabilities accessible even when direct questioning failed.