Chain-of-Thought Prompting (2022) › The Problem

Large language models trained on vast text corpora had acquired impressive knowledge and pattern recognition abilities, but they struggled with tasks requiring sequential reasoning or multi-step problem solving. When faced with complex questions, models often attempted to answer directly without breaking problems into manageable steps. This limitation became particularly evident on mathematical word problems, logical reasoning tasks, and planning problems where humans naturally decompose challenges into intermediate steps.

The standard few-shot prompting approach demonstrated this limitation clearly. Models were provided with input-output examples but not shown how to reason from inputs to outputs. When given a problem like "A farmer has 17 sheep. All but 9 die. How many are left?", models might try to compute $17 - 9 = 8$ directly, missing the logical reasoning that "all but 9" means 9 remain. Without explicit reasoning examples, models struggled to recognize when problems required careful step-by-step analysis rather than direct computation.

Mathematical word problems highlighted the issue particularly well. Problems involving multiple operations, sequential dependencies, or complex relationships between quantities often confused models that attempted to solve them in single steps. A problem requiring several calculations in sequence would fail if the model couldn't identify the intermediate steps needed. Models might know how to perform individual operations but couldn't orchestrate multiple operations in the correct sequence without explicit guidance.

Logical reasoning tasks suffered from similar limitations. Problems requiring deduction, inference, or hypothesis testing often produced incorrect results when models attempted immediate answers. Models might understand individual logical rules but struggle to chain multiple logical steps together to reach conclusions. Without examples showing how to break down logical problems into step-by-step deductions, models couldn't demonstrate the reasoning capabilities they possessed.

The brittleness of direct answering also manifested in how models handled ambiguous or context-dependent problems. Questions that required understanding implicit assumptions or working through scenarios systematically often produced incorrect or inconsistent results. Models might produce different answers to logically equivalent problems phrased differently, or fail to recognize when multiple reasoning paths needed to be explored before selecting the correct approach.

Additionally, the lack of explicit reasoning made it difficult to understand why models produced specific answers. When models gave incorrect responses, researchers couldn't identify where the reasoning process broke down because no intermediate steps were generated. This opacity hindered both understanding model capabilities and improving performance through better prompting strategies.