Reasoning Frontiers › Open Research Questions

The reasoning frontier moves quickly, but several fundamental questions remain open. These are not engineering questions that will be solved by more compute or better software. They are deep questions about how reasoning works, what current models are doing, and what would need to change to push further.

The most fundamental open question in reasoning model development is: how do you build reliable verifiers? The effectiveness of test-time compute scaling depends critically on the ability to evaluate reasoning quality, either for selecting from N samples or for providing training signal via RL.

In narrow domains like mathematics and code, automated verification is feasible. For open-ended tasks, it is not. Building models that can reason well about scientific hypotheses, strategic plans, or ethical dilemmas requires verifiers capable of evaluating complex, multi-dimensional quality. Current approaches each have significant limitations.

LLM-as-judge uses a separate language model to evaluate reasoning quality. The problem is that the judge model may have similar blind spots to the model being evaluated. If both models are trained on similar data, they may share systematic errors. Worse, the model being evaluated may learn to produce outputs that fool the judge without improving at the task, a form of reward hacking that is difficult to detect.

Human evaluation is accurate but expensive and not scalable to the volume of training examples needed for RL. You cannot build strong reasoning models using only human evaluation as the reward signal unless you have access to an enormous pool of expert human evaluators.

Constitutional principles involve having models self-evaluate against explicit criteria. This is effective for some dimensions of quality but gameable: a model that learns that longer responses with more hedge words receive higher constitutional scores will optimize for those surface features, not for actual quality.

The reward hacking concern is directly relevant here. A model trained to maximize a specific reward signal will, given sufficient capacity and training, find ways to achieve high reward that do not correspond to reasoning quality. Examples include generating plausible-looking but incorrect mathematical proofs that fool the verifier, producing code that passes test cases by exploiting edge cases in the test suite rather than solving the underlying problem, and writing lengthy reasoning traces that include surface "reasoning" steps (stating what approach will be taken, restating the problem, checking edge cases that are not relevant) without logical progress.

This is an instance of Goodhart's Law: when a measure becomes a target, it ceases to be a good measure. The field does not yet have a reliable solution to this problem. The partial solutions that exist (diverse evaluation across multiple benchmarks, adversarial evaluation with held-out test cases, process-level evaluation rather than outcome-only) each address some instances of reward hacking but not all.

A related and deeper question is whether current reasoning models are learning general reasoning capabilities or are learning sophisticated pattern matching over a distribution of reasoning problems that happens to look like reasoning.

The distinction matters enormously for understanding the limits of current models. General reasoning would transfer to qualitatively new problem types not represented in training. Sophisticated pattern matching would not, or would do so only to the extent that the new problem type shares surface features with the training distribution. Evidence on both sides exists.

Evidence for general reasoning includes o3 achieving over 75% on ARC-AGI, a benchmark explicitly designed to test novel abstraction. DeepSeek-R1-Zero developed self-correction without explicit supervision, suggesting a capability that was discovered rather than memorized. Models trained on mathematics show improvement on code reasoning and vice versa, suggesting that something is transferring beyond domain-specific patterns.

Evidence against general reasoning includes systematic failures on adversarial modifications of seemingly mastered problem types. If you take an AIME problem and change the numbers in a way that invalidates a common solution strategy, a model that appears to "understand" the original problem often fails on the modified version. The ARC-AGI score of 75%, while impressive, still leaves a significant gap and was achieved at very high compute cost with additional training on ARC-style problems. Models sometimes produce convincing-looking but incorrect reasoning that convinces human readers, a failure mode that would not occur with systems that reason reliably.

The current consensus is that something real is happening in these models: capabilities beyond pure memorization that generalize at least partially to new problem types. But the nature and limits of that capability are not well characterized, and it is probably not the same as human reasoning. It is something new, with its own strengths and weaknesses that do not map cleanly onto either "general reasoning" or "sophisticated pattern matching."

The DeepSeek-R1 paper documented the spontaneous emergence of self-correction behavior during RL training. This emergent behavior, not explicitly trained but arising from outcome reward optimization, raises deep questions about what other capabilities might emerge given sufficient scale and training compute.

The training dynamics of RL on reasoning tasks are not well characterized. Current training procedures are largely empirical: try something, see what happens, adjust based on what was observed. There is no theoretical framework that predicts which capabilities will emerge, when they will emerge, or what training conditions are necessary and sufficient to produce them.

Questions that remain open include: why does self-correction emerge specifically from outcome-reward RL, rather than from supervised fine-tuning on reasoning traces? What training signal distinguishes the regime where self-correction is reinforced? Are there other reasoning capabilities, such as systematic planning or structured search, that will emerge at scale? Can we predict which capabilities will emerge before training, and if so, what would that prediction look like? How do emergent capabilities interact with each other and with the base model's existing knowledge?

Understanding these dynamics is both scientifically important and practically important. Scientifically, it could tell us something fundamental about the relationship between reward optimization and the development of cognitive capabilities. Practically, it would allow more targeted training procedures: instead of training and hoping for emergent capabilities, you would know what to do to produce specific capabilities.

Early RL-based reasoning training required substantial compute and large datasets of problems with known solutions. DeepSeek-R1-Zero showed that pure RL can produce strong reasoning capabilities even without a carefully curated supervised warmup set. This raises the question of how much the quality and diversity of training problems matters relative to the scale of RL training.

Current evidence suggests that both matter, but their relative importance is unclear. More diverse problem sets seem to produce more generalizable reasoning, suggesting that covering many different reasoning patterns during training is valuable. But the marginal value of problem diversity versus raw scale (more compute on a smaller, more focused problem set) is not quantified.

A related question concerns curriculum: whether presenting harder and harder problems as training progresses leads to better reasoning than training on a diverse mixture from the start. Human education relies heavily on curriculum, building from simple to complex, and there is intuitive appeal to the idea that language models might also benefit. Some evidence supports this, but the design of effective curricula for reasoning RL training is not well understood.

Current reasoning models are most impressive on formal domains where the required knowledge is contained in the problem statement. Mathematical proof problems contain the axioms and definitions needed to solve them. Competitive programming problems contain the complete specification. The model does not need to retrieve external information; it needs only to reason about what is given.

They are substantially less impressive on tasks requiring integration of factual knowledge with reasoning. A model asked to reason about a complex historical counterfactual needs both accurate historical knowledge and reasoning capability. If the factual knowledge is wrong due to hallucination or training data gaps, even correct reasoning produces wrong conclusions. The model cannot reason its way to the right answer if it starts from false premises.

This interaction between knowledge retrieval and reasoning remains an important open problem. Retrieval-augmented generation (RAG) addresses part of this by grounding reasoning in retrieved documents. But the integration of extended reasoning with dynamic knowledge retrieval is an active research area. How should a reasoning model decide when its internal knowledge is sufficient versus when it should consult external information? How should retrieved information be integrated into an ongoing reasoning trace without disrupting the logical structure already built? These questions do not have satisfying answers yet.

A significant practical research question is whether models can learn to think efficiently, applying extended reasoning where it helps and defaulting to direct answers where reasoning would not improve quality.

Current reasoning models are relatively blunt instruments. o1 applies extended reasoning to essentially all queries, including trivial ones where a direct answer would suffice. Asking a reasoning model "What is the capital of France?" and getting a response preceded by thousands of tokens of internal deliberation is wasteful and slow. This pattern wastes compute and increases latency without providing any quality improvement.

Research directions for adaptive compute allocation address this from several angles. Difficulty estimation uses a fast classifier that predicts whether a given query benefits from extended reasoning, before committing to a full reasoning trace. Early exit strategies allow reasoning models to terminate deliberation when confidence is high, without completing a predetermined reasoning trace length. Cascaded models route easy queries to fast direct-answer models and hard queries to reasoning models. Budget-constrained inference trains models to produce their best answer given a specified token budget for reasoning, gracefully degrading on harder problems when the budget is tight.

The goal is a reasoning system that is not uniformly expensive but intelligently allocates its thinking time based on problem difficulty, achieving a better cost-quality profile than either a pure direct-answer model or a uniform-reasoning model.

Current reasoning models excel at problems that can be decomposed into steps within a single context window. Multi-step planning problems that require maintaining a coherent strategy across very long reasoning chains, or across multiple tool-use interactions, remain challenging.

Consider a research agent tasked with synthesizing a literature review on a new topic. This requires planning what to search for, executing searches, reading and extracting information from multiple papers, identifying gaps and contradictions across sources, and synthesizing a coherent narrative. Each step requires both knowledge and reasoning, and the overall task requires maintaining a coherent long-horizon plan across many individual actions.

Current reasoning models applied naively to this setting tend to lose track of the overall plan as the context grows, fail to integrate information across multiple retrieved documents effectively, and struggle with the meta-reasoning required to decide what information is relevant to collect. The fundamental challenge is that a fixed context window, even a very long one, is not well suited to managing long-horizon tasks where the relevant information accumulates and evolves over many steps.

This is not a purely reasoning problem. It intersects with agent architectures, memory systems, and retrieval. But reasoning capability is a central component, and improving reasoning models is expected to improve long-horizon task performance. The architecture of future systems capable of long-horizon reasoning will likely need to combine extended deliberation with structured memory and explicit planning mechanisms in ways that current systems do not.