Reasoning Limitations › Reasoning Limitations: Systematic Failures and Shortcuts

Modern language models can solve olympiad-level math problems, write working code, and explain difficult scientific concepts in plain language. These capabilities can give the impression of reliable, general reasoning. But this impression is deceptive. Beneath the surface, these models exhibit systematic failures that expose a deep gap between their impressive performance on standard benchmarks and the kind of reliable, flexible reasoning we associate with human intelligence.

This chapter examines the structural limitations of LLM reasoning. We are not cataloging random errors or edge cases. We are exploring systematic failures: patterns of breakdown that recur predictably across models and tasks. Understanding these patterns matters for anyone deploying LLMs in practice or working to improve them. The failures are not bugs to be patched but features of how these models learn and represent knowledge.

Before modern LLMs, the NLP community spent decades studying how neural models fail on natural language tasks. The findings from that era remain directly relevant today, now amplified in scale. Researchers studying early neural networks for textual entailment in the 2010s discovered that their models could achieve near-human accuracy on the standard benchmarks while making elementary logical mistakes on carefully constructed adversarial examples. These were not fringe failures. They were systematic. The models had learned statistical patterns that happened to align with correct answers on the training distribution, not the underlying logical structure of the task. When researchers deliberately broke those patterns, accuracy collapsed.

The same story has repeated itself at larger scale with LLMs. Models trained on trillions of tokens develop increasingly sophisticated statistical associations. Some of these associations approximate systematic reasoning quite closely. Others are sophisticated versions of the same heuristics their predecessors exploited, now harder to detect because the models are larger and the patterns are subtler. Understanding which is which requires deliberate, adversarial evaluation rather than standard benchmark testing.

The chapter is organized around four major themes. First, we examine spurious correlations, the tendency of models to latch onto superficial patterns in training data rather than learning the underlying logic. Second, we explore reasoning shortcuts, where models exploit statistical regularities to answer questions without reasoning through them. Third, we investigate systematic failure modes, the recurring categories of task where LLM reasoning breaks down reliably. Finally, we discuss robustness challenges: how small, semantically irrelevant changes to a problem can produce dramatically different model outputs.