Chain-of-Thought › Standard Prompting vs. Chain-of-Thought
Language AI HandbookAll chapters Chain-of-Thought › Standard Prompting vs. Chain-of-ThoughtOne section · Sections in this chapter3 / 16 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
To understand what CoT adds, it helps to see standard prompting and CoT prompting side by side on the same problem.
Standard few-shot prompting shows the model input-output pairs:
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
Each can has 3 balls. How many tennis balls does he have now?
A: 11
Q: The cafeteria had 23 apples. If they used 20 to make lunch and
bought 6 more, how many apples do they have?
A:
The model sees that the answer to the first question is 11 and learns to pattern-match toward a number. But it has no explicit record of how that number was derived. When the model generates an answer to the cafeteria question, it is working purely from statistical patterns: questions like this tend to have small integer answers, and the numbers in the question provide anchors. This works well when the required computation is simple and fits within a single inference step. For more complex problems, the failure mode is that the model produces a plausible-sounding number that is statistically consistent with the question format but arithmetically wrong.
Chain-of-thought prompting adds reasoning steps to each example:
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
Each can has 3 balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans × 3 balls = 6 balls.
5 + 6 = 11. The answer is 11.
Q: The cafeteria had 23 apples. If they used 20 to make lunch and
bought 6 more, how many apples do they have?
A:
Now the model sees that good answers include explicit arithmetic. When it generates an answer, it is likely to also generate reasoning steps, and those steps help it arrive at the correct final number. For the cafeteria problem, a CoT-conditioned model will typically produce "The cafeteria started with 23 apples. They used 20, leaving 23 - 20 = 3 apples. Then they bought 6 more: 3 + 6 = 9 apples." The reasoning is explicit, verifiable, and structured.
The performance gap between these two formats grows with problem difficulty. On simple one-step arithmetic, both approaches work comparably. On multi-step word problems or questions requiring multiple inference steps, CoT substantially outperforms standard prompting, and the advantage widens as problem complexity increases. Wei et al. reported accuracy improvements of over 40 percentage points on the GSM8K grade-school math benchmark for large models, a gap that no amount of prompt engineering on the answer-only format could close.
A less obvious benefit of the explicit reasoning format is that wrong answers become diagnosable. With standard prompting, you see "17" when the answer is 9, and you have no idea whether the model misread the problem, performed incorrect arithmetic, or simply pattern-matched to a wrong number. With CoT, you see the step where things went wrong. This turns debugging from a black-box exercise into something resembling reading a student's test paper.