Reasoning Limitations › Summary

Reasoning limitations in LLMs are systematic, not random:

• Spurious correlations arise when models learn surface-level statistical associations in training data rather than underlying logical relationships, leading to strong in-distribution performance but poor generalization to distribution-shifted tests. Models learn what is correlated with correct answers in training, not what causes them.
• Reasoning shortcuts allow models to arrive at correct answers through heuristics like lexical overlap, positional biases, template matching, and hypothesis-only patterns, without performing the required inference. Contrast sets and invariance tests expose these shortcuts by disrupting the surface features while preserving the logical structure.
• Systematic failure modes include consistent weaknesses in negation, counting, multi-hop reasoning, constraint satisfaction, and compositional generalization. These failures are structural, arising from the mismatch between statistical next-token prediction and formal computation, rather than from insufficient scale.
• Robustness failures manifest as sensitivity to rephrasing, distractors, prompt format, and answer ordering. A robust reasoner should produce consistent outputs under semantically neutral transformations. LLMs often do not, revealing that their outputs depend on surface features rather than stable internal representations.
• Calibration failures mean that model confidence does not reliably track accuracy, with LLMs frequently overconfident in high-confidence ranges, making it difficult to know when to trust the model's outputs.
• Mitigation strategies include adversarial augmentation, neurosymbolic integration, chain-of-thought prompting, self-consistency sampling, and calibration techniques. None eliminates these limitations entirely, but each reduces their practical impact in specific settings.

As reasoning capabilities in LLMs continue to advance, the evaluation frameworks developed to measure these limitations will remain essential. Scaling may reduce some failure rates, but without targeted interventions, many of the structural patterns described here will persist. The next chapter on reasoning frontiers explores active research directions aimed at addressing these fundamental challenges.