Chain-of-Thought › Chain-of-Thought Prompting

When you ask a language model a hard question, it has two choices. It can jump straight to an answer, pattern-matching against whatever surface features most resemble the question. Or it can work through the problem step by step, articulating intermediate reasoning before committing to a conclusion. Chain-of-thought (CoT) prompting is a technique that coaxes models into the second behavior. By showing a model examples where reasoning is explicit and sequential, or simply by asking it to "think step by step," you can dramatically improve performance on tasks that require multi-step logic.

The core insight is deceptively simple: language models generate text token by token, and the tokens they produce become context for subsequent tokens. When a model writes out an intermediate reasoning step, that step is available as context when it generates the next step. The scratchpad directly shapes what the model can compute.

Chain-of-thought prompting was introduced by Wei et al. (2022) in a landmark paper demonstrating that large models given reasoning-rich few-shot examples substantially outperformed those given only input-output pairs, particularly on arithmetic, commonsense, and symbolic reasoning tasks. The improvement was striking enough to reframe how researchers thought about in-context learning: the format of examples matters as much as the examples themselves.

Before CoT, scaling up models was the dominant strategy for improving reasoning performance. You could throw a harder question at a bigger model, and bigger models tended to do better. CoT changed the equation by showing that how you prompt matters just as much as how large your model is. A well-designed CoT prompt on a 100-billion-parameter model can outperform a poorly designed prompt on the same model by a wider margin than simply doubling the model size would produce. That realization transformed prompt engineering from a folklore art into an active research discipline.