Reasoning Foundations › Reasoning Evaluation

How do we measure reasoning capability in LLMs? This is a harder problem than it might appear, and the field has developed increasingly sophisticated approaches in response to lessons learned from earlier benchmarks. The history of reasoning evaluation is partly a history of benchmarks being constructed, models achieving high scores on them, and then the community discovering that the high scores reflect something other than the intended capability.

The evaluation of reasoning in LLMs began with standard multiple-choice and short-answer benchmarks. These include datasets designed to test a broad range of knowledge and reasoning capabilities:

• MMLU (Massive Multitask Language Understanding): Tests knowledge and reasoning across 57 academic subjects including mathematics, science, law, and humanities. Each question is multiple-choice with four options. Introduced by Hendrycks et al. (2020), MMLU was originally a strong discriminator between model sizes. MMLU scores are now saturating as frontier models approach ceiling performance, with some models exceeding 90% accuracy on the full test.
• ARC (AI2 Reasoning Challenge): A dataset of grade-school science questions requiring reasoning about scientific concepts. Split into an Easy and a Challenge set, the challenge questions were specifically designed to be hard for retrieval-based systems, requiring application of knowledge rather than simple lookup.
• HellaSwag: Commonsense reasoning about how activities proceed. Models must choose the most plausible continuation of a scenario. Designed by Zellers et al. (2019) to be difficult for models that rely on statistical co-occurrence patterns; the wrong answers were generated by a language model and specifically filtered to be hard to distinguish from correct answers.
• WinoGrande: A large-scale Winograd schema challenge designed to require commonsense reasoning for pronoun resolution. Pronoun references are ambiguous without reasoning about context, and the dataset was constructed to minimize the effectiveness of simple co-occurrence statistics.
• BIG-Bench: A massive collaborative benchmark with over 200 tasks spanning diverse reasoning modalities, including logical deduction, analogical reasoning, mathematical reasoning, and more. Designed to be difficult enough to remain discriminative for several model generations.

Each of these benchmarks represents a significant community investment in building evaluations that resist spurious-correlation shortcuts. And yet each has also been found, in various studies, to admit partial exploitation through non-reasoning strategies, illustrating how difficult it is to construct reasoning benchmarks that cannot be gamed.

Mathematical reasoning deserves special treatment because math has clear right answers and requires explicit multi-step reasoning. Key benchmarks include:

• GSM8K (Grade School Math 8,000): A dataset of grade-school math word problems requiring multi-step arithmetic, released by Cobbe et al. (2021). Each problem has a human-written solution chain. Became a standard benchmark for evaluating multi-step arithmetic in LLMs. State-of-the-art models now approach near-perfect performance on GSM8K, prompting the development of harder alternatives.
• MATH: A dataset of competition mathematics problems at much higher difficulty, covering algebra, geometry, number theory, calculus, and combinatorics, released by Hendrycks et al. (2021). State-of-the-art models still fall short of human expert performance on the hardest problems, particularly those requiring novel proof strategies. MATH has remained a useful discriminator even as models have improved.
• MGSM: Multilingual versions of GSM8K, testing whether reasoning capability generalizes across languages. Multilingual evaluation is important because it probes whether the model has learned the underlying mathematical structure or merely the English-language conventions of arithmetic word problems.
• AIME and AMC: Problems from American high school mathematics competitions, which require substantially more sophisticated mathematical reasoning than GSM8K. A small number of problems from AIME are now used to evaluate frontier models' mathematical capabilities at the high end.

The trajectory of performance on mathematical benchmarks illustrates the scaling of reasoning capability clearly. GPT-3 achieved around 5% on GSM8K; GPT-4 achieved above 90%. The MATH benchmark saw similar progression over a shorter period. This rapid improvement has driven the construction of ever-harder benchmarks to maintain discriminative power.

• LogiQA: Logical reasoning questions drawn from Chinese civil service exam reading comprehension sections. Requires multi-step logical inference to determine correct answers among distractors designed to test for common fallacies.
• ReClor: Logical reasoning questions from standardized tests like GMAT and LSAT, which were designed to test the kind of logical reasoning required for graduate education. Provides a challenging and ecologically valid test of logical inference.
• FOLIO: A dataset requiring formal first-order logic reasoning, with premises and conclusions expressed in natural language and paired with formal logical annotations. Tests whether models can reason correctly in strictly formal contexts, not just informal ones.
• StrategyQA: Questions that require implicit multi-step reasoning strategies to answer, where the reasoning steps are not explicitly provided. The model must decompose the question, identify the sub-questions it needs to answer, and integrate the sub-answers correctly.

Each of these benchmarks targets a specific facet of reasoning capability. Using multiple benchmarks together provides a more complete picture of a model's strengths and weaknesses than any single benchmark can.

A recurring challenge in reasoning evaluation is that benchmarks get saturated. As models improve, they achieve ceiling-level performance on benchmarks that were originally quite difficult. Once a benchmark is saturated, it loses discriminative power: it cannot distinguish between models of different capability levels, and further improvement on that benchmark is not meaningful.

The saturation of MMLU and many initial reasoning benchmarks has driven the development of harder evaluation suites. MMLU-Pro, introduced in 2024, extends MMLU with harder questions and more answer choices to restore discriminative power. BIG-Bench Hard focuses on a subset of BIG-Bench tasks where models still perform poorly. The progression illustrates that the field faces a moving target: a good evaluation benchmark must be calibrated to the current frontier of model capability, but frontier capability is advancing rapidly.

This cycle, of creating benchmarks, watching them saturate, and creating harder ones, is itself informative: it suggests that the underlying capabilities are improving, but it also makes it difficult to track progress on a stable axis. Different research groups may report results on different benchmarks, making direct comparisons difficult and progress measurement noisy.

The saturation problem is also philosophically interesting. When frontier models approach ceiling on grade-school math benchmarks, this does not mean they have mastered grade-school math in the way a human understands mastery. A human who has mastered addition can apply it reliably in any context, in any format, under any surface variation. A model near ceiling on a specific benchmark may still fail at isomorphic problems presented unusually. Saturation of benchmark performance is not the same as complete, robust mastery.

A specific and serious challenge for reasoning benchmarks is data contamination: the possibility that benchmark examples, or examples very similar to them, appear in the model's training data. If the model has seen the benchmark problems (or problems with the same answers) during training, then high benchmark scores reflect memorization, not reasoning.

Contamination is difficult to detect because training datasets are large, poorly documented, and often not publicly inspectable. Several studies have found evidence of contamination in widely used benchmarks, casting doubt on some published results. Golchin and Surdeanu (2023) developed methods to detect contamination by testing whether models exhibit unusual memorization behavior on benchmark examples relative to paraphrased versions of the same problems. Their findings suggested nontrivial contamination in several widely evaluated models.

This has driven the creation of dynamic benchmarks (where new problems are generated at evaluation time) and of held-out test sets where the release date of problems post-dates the training cutoff of models being evaluated. The LiveBench project, for example, curates reasoning problems from recent competitions and publications, making sure that the problems could not have been in training data. The FrontierMath benchmark uses problems from ongoing mathematical research, which are even less likely to have appeared in training corpora.

Contamination concerns also affect the interpretation of capability improvements over model generations. If a newer model achieves higher scores partly because of contamination and partly because of real capability improvement, comparing raw scores across generations overstates the real improvement. Researchers have developed statistical tests for contamination, but none are definitive, and the field must live with this source of uncertainty.

A deeper challenge in evaluating reasoning is distinguishing between correct outcomes and correct processes. A model might arrive at the correct answer through a flawed reasoning process (for example, by making two mistakes that cancel out), or might have a logically valid reasoning process that happens to end at a wrong answer because of a factual error.

For many downstream applications, correct outcomes are what matter. But for understanding model capabilities and limitations, process evaluation is essential. A model that reliably produces correct answers through unreliable processes will fail unpredictably when conditions change slightly. A model that produces correct answers through reliable processes will generalize more predictably. Understanding the process is therefore important for engineering reliable systems, not just for scientific understanding.

This has led to the development of datasets that include step-by-step annotated solutions, against which model-generated reasoning chains can be compared. Evaluating whether each step in a chain is logically valid is, however, itself a challenging problem, one that current automated evaluation methods handle imperfectly. Manual evaluation of reasoning chains is expensive and slow, which limits the scale at which process evaluation can be conducted.

Uesato et al. (2022) developed outcome reward models and process reward models as complementary evaluation approaches. Outcome reward models simply check whether the final answer is correct. Process reward models evaluate each step in the chain, giving richer feedback but requiring annotated step-by-step solutions for training. The process reward approach has been shown to improve performance on multi-step reasoning tasks when used for training or inference-time selection, validating the intuition that step-by-step correctness is a useful training signal.

A recent development is using LLMs themselves as evaluators of reasoning quality. A judge model is given a reasoning problem, the reference solution, and a candidate solution, and asked to rate the candidate's reasoning quality. This approach scales better than human evaluation and can be applied to open-ended tasks where there is no single correct answer.

The challenge is that LLM-based evaluation inherits the same failure modes as LLM reasoning. A judge model may be fooled by confident-sounding but incorrect reasoning chains, may have systematic biases toward longer or more elaborate explanations, and may not reliably catch subtle logical errors. Calibration of LLM judges against human expert judgments is an active area of research. Zheng et al. (2023) showed that GPT-4 as a judge has substantial agreement with human preferences on many tasks but exhibits systematic biases, including a tendency to prefer longer responses and to be anchored by the order in which responses are presented.

A practical approach to mitigating these biases is to use multiple judges, present candidates in random order, and check for consistency across multiple evaluation runs. Even with these precautions, LLM-based evaluation is better treated as a fast, scalable approximation to human evaluation than as a substitute for it on high-stakes evaluation tasks.