Capability Frontiers: Emerging AI, World Models, Planning

For most of the history of artificial intelligence, the challenge was getting systems to do anything useful at all. Researchers spent decades hand-crafting rules, designing specialized architectures, and training narrow systems on carefully curated benchmarks. Progress was real but incremental, and each advance felt like chipping away at a vast problem rather than solving it.

Then something shifted. Language models grew larger, trained on more data with more compute, and began exhibiting capabilities that no one had explicitly programmed. Models started reasoning about problems they had never seen before, generating coherent arguments, writing functional code, and solving mathematical puzzles. These emergent behaviors raised a fundamental question that now sits at the center of AI research: what exactly can these systems do, and what are the limits of what they can learn to do?

This question is harder to answer than it seems. When a model correctly solves a complex math problem, does that reflect mathematical reasoning that generalizes beyond memorized patterns or an extraordinarily sophisticated form of pattern completion? When it writes working code from a vague natural language description, is it reasoning about program semantics or retrieving learned patterns from millions of similar programs in its training data? The answer may be "both," and distinguishing between them matters enormously for understanding what these systems are and where they are headed.

This chapter examines the frontier of language model capabilities. We will look at where models currently excel, where they still struggle, and what research directions are expanding the boundaries. The goal is not to catalog every benchmark result but to understand the deeper patterns: what enables certain capabilities, why others remain elusive, and what emerging research suggests about where these systems are headed. Building on the attention mechanisms and transformer architectures covered in earlier parts of this book, we now turn to the question of capability: what can a system trained on language learn to do?

One of the most striking findings in modern AI research is that some capabilities appear not gradually but abruptly as models scale. These emergent capabilities exist as though a threshold must be crossed before a behavior becomes reliable. Below a certain scale, the model produces noise or random guesses. Above it, performance jumps to useful or even expert-level quality.

The term "emergence" here carries a specific technical meaning. A capability is emergent when it cannot be predicted by extrapolating from performance at smaller scales, and when the behavior appears qualitatively different rather than just quantitatively better. The claim is that an apparently new behavior appears, rather than merely improving faster.

This definition requires some care. Some researchers have argued that apparent emergence can be an artifact of nonlinear evaluation metrics rather than a true discontinuity in underlying capability. If you measure performance with a threshold metric (does the model get the answer exactly right or not?), small continuous improvements in the underlying latent ability can produce a sudden jump in measured accuracy once the latent ability crosses the correctness threshold. This is an important methodological concern, and it motivates using smooth metrics alongside binary ones when evaluating capability curves. Nevertheless, certain behaviors do appear reliably only above certain scales, and whether the emergence is "real" at the latent level or a measurement artifact, the practical implication is the same: these behaviors are not available at small scale and become available at large scale.

Perhaps the most discussed emergent capability is in-context learning: the ability to adapt to new tasks given only a few examples in the prompt, without any gradient updates to the model's weights. You provide a handful of input-output pairs, and the model generalizes to new inputs in the same format.

What makes this striking is that the model is not being trained. Its weights are frozen. Yet somehow, reading a few examples activates the right behavior. The model appears to recognize the task structure from the examples and apply it to new inputs. This is a fundamentally different kind of learning from gradient-based fine-tuning: the "update" happens entirely within the forward pass.

Mechanistic interpretability research has offered one hypothesis for how this works: large models learn an implicit learning algorithm during pretraining. When given examples, the model essentially runs gradient descent in the forward pass, adjusting its internal representations to fit the demonstrated pattern without updating any weights. This framing is speculative but has been supported by some theoretical work showing that transformers have the computational structure to implement gradient-based learning. The key observation is that attention heads can implement the update rule for a linear model, with keys and values playing the roles of stored data points and queries playing the role of new inputs.

A simpler hypothesis is that in-context learning is a form of retrieval: the model retrieves relevant knowledge and patterns from its pretraining distribution, conditioned on the examples provided. Under this view, the examples are not teaching the model anything new but rather narrowing the space of behaviors it considers appropriate. Both hypotheses may be partially right, and the mechanism may differ depending on the task and model size.

The practical implications are significant regardless of mechanism. In-context learning allows a single model to serve many tasks without fine-tuning, reducing the engineering overhead of deployment. It also raises questions about what "knowing" a task means for a language model: if you can specify a task entirely through examples, the boundary between "tasks the model knows" and "tasks the model doesn't know" becomes blurry.

Performance on in-context learning tasks scales with both the number of examples and the model size. More examples help more for larger models, suggesting that smaller models lack the capacity to extract and use pattern information from long contexts as effectively. The quality of the examples also matters substantially: ambiguous, inconsistently formatted, or incorrectly labeled examples degrade performance more than having fewer high-quality examples.

A second emergent capability involves reasoning chains. When prompted to show its work, a sufficiently large model can break a problem into steps, work through each step, and arrive at a more reliable answer than if asked to respond immediately.

This capability, called chain-of-thought prompting, was discovered empirically and has since been extensively studied. The pattern is consistent: below a certain model size (roughly 100 billion parameters in some studies), chain-of-thought prompting does not help or can even hurt performance. Above that threshold, it substantially improves accuracy on multi-step reasoning tasks. The reason smaller models fail to benefit is not entirely clear, but one hypothesis is that they lack the parametric knowledge required to successfully execute the intermediate steps, so showing the steps only exposes errors rather than reducing them.

The mechanism behind this improvement involves several factors. By externalizing intermediate steps, the model can check its work at each stage and course-correct rather than letting early errors compound silently. The chain of reasoning also allows the model to use the context window as a kind of scratchpad, storing partial results that would otherwise need to be compressed into a single hidden state. This is especially valuable when problems require tracking multiple interacting quantities or when the final answer depends on applying transformations in a specific order.

There is a deeper point here about what chain-of-thought does cognitively. Human experts who work through problems on paper are not just showing their work for an audience; the external representation is a cognitive prosthesis. By offloading intermediate results to paper, they free up working memory for higher-level reasoning. The context window appears to serve an analogous function for language models: externalizing state that would otherwise need to fit in the finite-dimensional hidden state.

Extensions of this idea include several variations that address different failure modes. Self-consistency samples multiple independent reasoning chains and takes a majority vote on the final answer, reducing the impact of any single reasoning path going astray. Tree-of-thought explores multiple candidate reasoning paths simultaneously, allowing the model to evaluate and prune branches that seem unpromising before they have been fully developed. Least-to-most prompting decomposes a hard problem into a sequence of progressively easier subproblems, with each subproblem's answer becoming context for the next, preventing the model from having to bridge too large a reasoning gap in a single step.

Each of these extensions builds on the same core insight: externalizing intermediate computation improves reliability, and structuring how that computation is externalized further improves it.

Early language models were trained to predict the next token, not to follow instructions. They would complete a prompt but had no notion of what a user was asking for. A model asked "What is the capital of France?" was as likely to respond with "What is the capital of Germany?" (completing a pattern of geography quiz questions) as with "Paris." A significant capability shift came when models were trained on human feedback to produce responses that matched human intent, a process called reinforcement learning from human feedback (RLHF).

The resulting models display dramatically improved instruction following. They can interpret ambiguous requests, apply implicit constraints, maintain conversation context across turns, and adjust their behavior based on user feedback within a conversation. This shift transformed language models from completion engines into assistants.

What makes instruction following interesting from a capability standpoint is how general it is. A model that learns to follow instructions on thousands of task types during fine-tuning appears to develop a general notion of intent. Rather than memorizing what to do for each task type, the model learns to model what the user wants. This general intent-following capability transfers to novel task types that were not explicitly in the fine-tuning data.

The development of instruction following as a capability is tightly connected to the development of alignment as a field. When a model follows instructions reliably, it is also a model that could in principle follow harmful instructions just as reliably. Understanding where the limits of instruction following are, and how to set those limits appropriately, is one of the central challenges in deploying these systems responsibly. We will return to this in the chapters on alignment; for now the important point is that instruction following as a capability is separable from alignment: a model can be highly capable at following instructions and poorly aligned, or less capable but better aligned.

A more recent class of emerging capabilities involves tool use: the ability to call external functions, search the web, execute code, or interact with databases. Models trained to use tools do not merely generate text describing what they would do. They invoke functions and process results.

This capability sits at the border between language understanding and agency. A model that can search for current information, run code to verify a computation, and synthesize results from multiple sources is doing something qualitatively different from text generation. It is operating on the world, not just describing it. The output of the model is no longer just text but a sequence of actions whose consequences extend beyond the context window.

The computational requirements for tool use are interesting to analyze. The model must represent the tool's interface (what arguments it takes, what it returns), decide when to invoke it, formulate a valid call, parse the result, and integrate it with prior context. Each of these steps involves a kind of structured reasoning that earlier models struggled with. That modern models can perform this reliably, even across unfamiliar tools described only in documentation, suggests the development of some general understanding of function interfaces as an abstract concept.

Tool use also enables a form of self-verification that pure text generation cannot achieve. A model that writes code and executes it gets immediate, reliable feedback. A model that performs a calculation and checks it with a calculator gets a ground-truth answer. This grounding to external reality makes tool-augmented agents more reliable in their specialized domains than models operating purely from internalized knowledge.

A central debate in language AI concerns whether large language models develop anything resembling a world model: an internal representation of how things work that goes beyond pattern matching over text.

The question matters practically. A model with a coherent world model should generalize to novel situations in ways that a pure pattern-matcher cannot. If a model has learned that hot objects burn skin because heat transfers energy to biological tissue, it should be able to reason correctly about a novel scenario involving a hot object it has never seen mentioned in training data. If it has instead learned a surface correlation between "hot object" and "pain/injury" text, it may fail in contexts where the phrasing differs from its training distribution.

This distinction, world model versus surface statistics, is not just philosophical. It predicts different failure modes, different generalization curves, and different responses to distribution shift. Understanding which one we have matters for knowing what these systems can reliably be asked to do.

There is substantial evidence that language models encode factual world knowledge implicitly in their weights. Models trained purely on text can answer questions about physical reality, spatial relationships, and causal chains with remarkable accuracy. They know that ice melts when heated, that cities have populations, and that dropping an object causes it to fall.

This knowledge is clearly absorbed from text rather than from direct experience. But the question is whether it exists as a retrievable fact or as something more structured. Probing studies have found that model representations encode geometric relationships between concepts, beyond co-occurrence statistics. Words for hot and cold sit at opposite ends of a temperature axis in the model's embedding space. Words for large and small are similarly structured. Concepts organized by the model are often in sensible positions relative to concepts related by physical law, taxonomic hierarchy, or temporal order. This suggests that models develop something like a semantic geometry of the world, rather than a lookup table.

Recent interpretability work has gone further, finding evidence that models develop abstract representations of specific domains. One prominent finding involves spatial reasoning: models trained on natural language appear to develop implicit 2D maps when processing stories about characters moving through physical spaces. The coordinates of these internal maps correlate with spatial positions in the described scene, suggesting a form of structured world representation rather than surface-level pattern matching. The model is apparently building an internal spatial model of the story and querying it to answer questions about character positions, even though no spatial information was ever explicitly encoded in training.

Similar findings have emerged for other domains. Models appear to track entity states through narratives, representing whether objects are open or closed, full or empty, inside or outside other objects. When a story describes a character opening a box and then placing something inside, the model's internal representation appears to update accordingly, consistent with tracking the state of the world rather than just the sequence of words.

Despite this evidence, language models face a significant challenge called systematicity. A system with a compositional world model should be able to compose known concepts in novel ways and apply rules consistently. If you know what a red cube looks like and you know what a blue sphere looks like, you should be able to correctly answer questions about a blue cube you have never seen described.

Language models often struggle with this kind of compositional generalization. They perform well on problem types seen in training and poorly on held-out combinations of familiar components. A model that correctly answers "What is the capital of France?" and "What is the largest city in France?" may give inconsistent answers to "If Paris is the capital of France, what follows about France's largest city?" when the question is framed in a way that requires explicitly composing both facts.

This asymmetry suggests that some of the apparent world knowledge in these models is shallow: they have learned correlations that happen to produce correct answers on typical inputs but break down when the combination is novel. The model has stored "Paris is the capital of France" and "Paris is the largest city in France" as separate facts, rather than as constraints in a coherent world model that would automatically enforce consistency between them.

The debate about whether this reflects a fundamental limitation or a training data coverage problem is ongoing. Models trained on more diverse and adversarially constructed data show better compositional generalization, suggesting the ceiling is not purely architectural. At the same time, there are theoretical arguments that transformers have representational limitations that make systematic generalization harder than it is for symbol-manipulating systems. A transformer processes sequences through learned statistical associations, while symbolic systems manipulate structured representations according to explicit rules. These two approaches may have complementary strengths: transformers excel at fuzzy pattern matching over natural language while symbolic systems excel at exact rule application over formal structures.

Causal reasoning represents a specific challenge for language models, and understanding why illuminates something fundamental about how they work. Correlation-rich text data encodes causes and effects together, but the directionality of causation is often implicit. Text about a bridge collapsing after stress fractures appear describes a causal sequence, but a model trained purely to predict text may capture the co-occurrence pattern without representing the causal direction. When asked "If the stress fractures had been repaired, would the bridge have collapsed?" the model must reason about a counterfactual, and that requires causal understanding rather than associative recall.

This matters for practical applications. A model advising on medical decisions, engineering designs, or business strategies needs to understand causal structure, not just correlations. Recommending an action because it correlates with good outcomes in training data is categorically different from recommending it because it causes good outcomes. Confusing these can lead to recommending harmful interventions that correlate with recovery (perhaps because they are given to patients who would have recovered anyway) rather than identifying the interventions that cause recovery.

Language models trained on text often exhibit this confusion. They can identify that two things tend to co-occur but have difficulty distinguishing whether A causes B, B causes A, or both are caused by a third factor C. Human reasoning about causation relies on interventional understanding: we know what would happen if we changed something, derived from physical models, causal graphs, or direct experimentation. Models trained on observational text lack this interventional grounding by default.

Recent work has explored ways to improve causal reasoning in language models. Exposure to structured causal data, such as text explicitly describing interventions and their effects, improves performance on causal reasoning benchmarks. Explicit training on counterfactual examples, where the model must reason about what would have happened differently, shows similar benefits. Prompting strategies that encourage causal framing ("What would change if X were different?") also help. Progress has been real but the gap to human-level causal reasoning remains large, suggesting this is an area where architectural or training innovations beyond prompt engineering may be needed.

The shift from language model to agent represents one of the most significant capability frontiers. An agent does not just answer questions: it pursues goals, takes actions, observes outcomes, and adjusts behavior accordingly. This requires capabilities that language models were not originally designed to exhibit.

Effective planning begins with decomposition: breaking a complex goal into manageable subgoals. When asked to "write a research report on climate change," an intelligent system should recognize that this requires gathering information, organizing themes, writing sections, and reviewing for coherence. These are distinct subtasks that can be executed sequentially, and each subtask is substantially simpler than the original task.

Modern language models have shown surprising capability for task decomposition when prompted appropriately. Given a complex goal, they can generate reasonable plans with appropriate granularity. They naturally produce hierarchical plans, identifying major phases before specifying substeps within each phase, which aligns with how humans approach complex projects.

The challenge comes in execution: pursuing the plan, handling unexpected obstacles, and adjusting when subgoals fail. A plan generated at the beginning of a task may become invalid partway through due to information discovered during execution. A model that can generate a good initial plan but cannot revise it intelligently when circumstances change is less useful than one that can adapt. This requires maintaining a representation of the current state of the task, tracking which subgoals have been completed and which remain, and identifying when the original plan needs revision.

Research on planning with language models has identified several failure modes worth understanding. Models sometimes generate plans that look locally reasonable but are globally inconsistent: one step requires resources that another step has already consumed, or two steps that cannot be executed simultaneously are not flagged as conflicting. They struggle with resource constraints when those constraints interact in non-obvious ways. They can be overly optimistic, generating plans that assume subgoals will succeed without accounting for failure cases or providing fallback strategies. And they sometimes produce plans with the wrong granularity, either too coarse to be actionable or too fine-grained to be strategically coherent.

A fundamental limitation for language model agents is working memory. The context window imposes a hard limit on how much information the model can hold in view at once. For short tasks, this is rarely a problem. For long-horizon tasks, where progress must be tracked over hundreds of steps, it becomes a significant constraint.

The context window functions as external working memory for the model: everything the model can consider simultaneously must fit within it. This is qualitatively different from human working memory, which is supplemented by long-term memory that can be recalled based on cues. For a human working on a long project, relevant context from earlier in the project is retrievable when needed; for a language model agent, earlier context is simply gone once it scrolls out of the context window unless explicitly saved and retrieved.

Various approaches have addressed this limitation. Retrieval-augmented generation allows models to query an external knowledge base rather than relying solely on the context window, effectively giving the model a form of long-term memory. Memory augmentation systems store summaries of past actions in external buffers that can be retrieved when relevant, preserving task-relevant context across context window boundaries. Hierarchical planning approaches use one model to generate high-level plans and another to execute individual steps, reducing the cognitive load on any single forward pass and allowing long-horizon tasks to be decomposed into shorter-horizon segments.

None of these approaches fully solves the long-horizon planning problem, but together they extend what agents can reliably accomplish. The current state of the art can handle tasks that take tens of steps to complete with reasonable reliability. Tasks requiring hundreds of interdependent steps remain challenging because context length is limited and errors accumulate over long horizons. A small probability of error at each step, multiplied over many steps, can quickly dominate overall reliability.

The question of how to build agents that remain reliable over long horizons is one of the active research areas in AI systems. Promising directions include planning algorithms that better handle failure that explicitly model uncertainty and plan for failure cases, better integration of symbolic planning tools that can enforce global consistency, and improved self-monitoring mechanisms that detect when the agent has drifted from its intended plan.

A capability closely related to planning is self-correction: the ability to evaluate one's own outputs, identify errors, and produce improved versions. This matters because language models make mistakes, and a model that can catch and fix its mistakes is more reliable than one that cannot. The ideal agent would be one that produces an initial output, checks it, finds errors, revises them, and iterates until the output meets quality criteria.

Self-correction turns out to be harder than it sounds. Studies have found that when models are simply asked to check their own answers, they often validate incorrect responses rather than catching errors. The same reasoning failure that produced the error often persists during review. This makes intuitive sense: if the model failed to reason correctly the first time due to some systematic bias or missing knowledge, reviewing the answer does not fix that underlying issue.

Effective self-correction seems to require some form of separation between generation and evaluation. Approaches that work better include providing explicit feedback from a trained critic model that evaluates outputs on specific criteria, rather than asking the generating model to evaluate its own outputs. Breaking review into smaller targeted checks rather than open-ended review also helps: "check whether the solution handles the edge case where the input is empty" is a more tractable review task than "check whether the solution is correct."

The most reliable form of self-correction involves grounding: checking outputs against external verifiable information. A model that generates code and then executes it gets concrete feedback about whether the code works, feedback that is unambiguous and not subject to the same reasoning errors that affected generation. A model that generates a mathematical claim and then checks it against a symbolic solver gets reliable feedback about correctness. This kind of grounded self-correction is substantially more reliable than pure language-based review and represents one of the strongest arguments for tool-augmented systems over pure language model agents.

Reflection, a broader capability, involves correcting specific outputs and learning from patterns of errors over time. An agent that notices it consistently fails on problems involving negative numbers might hypothesize a systematic weakness and actively seek to avoid such problems or compensate for them. This kind of meta-learning in context is an active research area, with approaches including explicit reflection prompts that ask the model to identify what went wrong and how to avoid it in the future.

Language models began as purely text-based systems, but modern architectures increasingly process images, audio, and video alongside text. This multimodal capability opens new possibilities but also introduces new failure modes.

The integration of vision and language in large models follows a pattern similar to language-only scaling: capable representations emerge from training on large datasets of image-text pairs, with quality improving as model size and data scale increase. The architecture typically involves encoding images into a sequence of visual tokens using a vision encoder (often a vision transformer or CNN), projecting those tokens into the language model's embedding space, and treating the combined sequence of visual and text tokens as input to a single transformer.

This design choice, treating visual tokens as "visual words" that the language model processes alongside text, has proved surprisingly effective. The language model applies the same attention mechanism to visual tokens as to text tokens, allowing visual and linguistic information to be integrated at every layer of processing rather than just at the final output stage.

Vision-language models can describe images, answer questions about visual content, read text in images (optical character recognition), and reason about spatial relationships between objects. At their best, they can interpret complex charts and diagrams, understand visual metaphors and idioms, and integrate visual and textual information to answer questions that neither modality alone could answer. A question like "Based on this photograph, what does the building's architectural style suggest about the period of construction?" requires both perceptual capabilities (recognizing architectural features) and linguistic/cultural knowledge (associating those features with historical periods).

The integration is not perfect, however. Models frequently struggle with fine-grained visual details: counting objects accurately in cluttered images, identifying subtle differences between similar objects (two slightly different dog breeds, two similar paint colors), or tracking the positions of multiple objects simultaneously. They can also hallucinate visual details with the same confident fluency that language-only models use when confabulating facts. A model asked about the number of people in a crowd may give a plausible-sounding but incorrect count, reported with the same confidence as accurate answers.

A recurring challenge is that vision-language models tend to rely heavily on language priors when visual information conflicts with expectations. If an image shows an unusual arrangement of familiar objects, the model may describe the expected arrangement rather than the actual one. A banana resting on a table that is itself balanced on the floor of a swimming pool might be described as "a banana on a table" without acknowledging the unusual context, because that is what the language prior expects. This reflects a fundamental tension in multimodal models: the language component has learned strong priors about how the world typically looks, and these priors can override direct visual evidence.

Spatial reasoning is a particular frontier for multimodal models. Humans effortlessly reason about 3D structure from 2D images, track object positions as viewpoints change, and perform mental rotations to understand how objects relate in space. These capabilities appear to be deeply connected to physical intuition developed through embodied experience of moving through and manipulating a three-dimensional world.

Current vision-language models handle simple spatial reasoning tasks reasonably well. Asking which of two objects is to the left, which is above another, or whether an object is inside or outside a container yields accurate answers in many standard cases. Performance degrades significantly on tasks requiring 3D understanding from 2D images, viewpoint invariance (recognizing the same scene from a different angle), or mental simulation of physical processes. Asking whether a certain object would be visible after rotating a scene, or which of two objects would fall first if a table were tilted, produces much less reliable answers than asking about color or shape.

This gap reflects a deeper challenge: spatial and geometric intuition may require a kind of grounded experience that text and image data cannot fully provide. Knowing what it feels like to be inside a room, to pick up an object, or to watch something fall gives humans intuitions about 3D space that are not easily extracted from static images. Some researchers argue that progress here will require models trained on simulated physical environments, allowing them to develop representations tied to the consequences of actions, not just to the appearance of scenes.

This has motivated a research direction called embodied AI, which trains agents in simulated environments where they can navigate, manipulate objects, and observe the consequences of their actions. The hypothesis is that exposure to interactive experience creates richer spatial representations than passive observation. Early results are promising, with embodied agents showing better performance on spatial reasoning benchmarks than equivalent models trained only on static data.

An important recent finding is that chain-of-thought reasoning extends to multimodal settings. When models are prompted to reason step by step before answering visual questions, accuracy improves substantially, with the improvement following a similar threshold pattern to language-only chain-of-thought. A model asked to interpret a complex chart performs better if it first describes what it sees ("There are three lines representing different model sizes; the y-axis shows accuracy and the x-axis shows training compute") and then answers the question, compared to answering directly.

This suggests that the underlying reasoning mechanism is not purely visual or purely linguistic but operates at a higher level of abstraction. The model can use language to scaffold visual reasoning, breaking a complex perceptual judgment into describable intermediate steps that it can work through sequentially. Language provides a medium for representing visual concepts in a form that the model's reasoning mechanisms can operate on more reliably.

Multimodal chain-of-thought is particularly valuable for tasks that mix visual and quantitative reasoning, such as reading a chart and then computing a derived quantity, or examining a diagram and then answering questions that require understanding spatial relationships. In these hybrid tasks, externalizing the reasoning process appears to reduce compounding errors significantly. The model is forced to make its perceptual judgments explicit before attempting the quantitative reasoning, creating checkpoints where errors in perception can be caught before they propagate into the reasoning phase.

Understanding where models stand requires careful measurement. The history of language model benchmarking is a history of rapid saturation: models exceed human performance on a benchmark that was designed to be challenging, and researchers must design harder benchmarks. This cycle has accelerated dramatically in recent years, with some benchmarks going from introduction to saturation in under two years.

This pattern reveals something important about how capabilities develop. Models often learn to exploit statistical regularities in benchmark data rather than developing the underlying capability the benchmark was designed to measure. When a benchmark is saturated, it means either that the capability has been demonstrated outside the benchmark, or that the benchmark was a poor proxy for it. Distinguishing between these cases requires careful analysis and often a second round of evaluation design.

Several major language benchmarks, including some designed to measure common-sense reasoning and reading comprehension, were saturated by large models within a few years of their release. Models scored at or above human performance levels, suggesting that the tasks were solved. In several cases, closer examination revealed that the models had likely seen the benchmark questions during pretraining, inflating their apparent performance through memorization rather than reasoning.

This data contamination problem is fundamental and difficult to solve completely. Pretraining datasets now encompass such a large fraction of the internet that any publicly available benchmark is potentially contaminated. A benchmark question posted on a forum, a paper describing the benchmark, or any other public mention can end up in the training corpus. Researchers have responded with approaches including held-out benchmarks that are never made public (their questions are evaluated through an API that returns only scores, not answers), dynamically generated benchmarks with different questions each evaluation, and benchmarks derived from sources unlikely to appear in training data (hand-crafted problems, recent unpublished work, or problems in languages underrepresented in training).

Detecting contamination is also difficult. One approach tests whether models know the correct answers but can also recall the specific phrasing of benchmark questions, suggesting memorization rather than understanding. Another approach compares performance on original benchmark questions versus paraphrased versions: memorized answers should survive paraphrase but reasoned answers should remain more stable. These detection methods are imperfect, and the field is still developing standards for contamination auditing.

A key methodological concern in capability evaluation is distinguishing reasoning that generalizes from pattern matching. A model that correctly answers 90% of a reasoning test may be applying transferable reasoning on some questions and matching training distribution patterns on others, and it is hard to tell which is happening on any given question.

Adversarial evaluation methods have been developed to probe this distinction. One approach generates new questions by modifying existing ones in ways that preserve the reasoning structure but alter surface features. If a question about "John and Mary" is modified to be about "Takeshi and Aiko" with otherwise identical structure, and performance drops, it suggests the model was relying on surface familiarity rather than solving the underlying reasoning problem. If performance is preserved, it is evidence that the reasoning transfers.

Another approach uses questions that require multiple interleaved reasoning steps, where the answer at each step feeds into the next. Pure pattern matching tends to fail on these tasks because the specific combination of steps is unlikely to appear exactly in training data. Even if the model has seen similar chains in training, the exact sequence with these specific values will typically be novel.

A third approach uses minimal pairs: two questions that differ in exactly one logically relevant way, such that the correct answers are different. A model that relies on patterns will apply the same pattern to both and get one wrong; a model that reasons correctly will differentiate them. This methodology has revealed surprising failures in models that score well overall: they may systematically fail on specific minimal pair types that expose reasoning weaknesses not visible in aggregate accuracy numbers.

Among the benchmarks that have driven recent capability research, the ARC-Challenge dataset (which tests scientific reasoning at a grade-school level but with questions deliberately selected to resist simple lookup and word matching) and MMLU (Massive Multitask Language Understanding, which covers 57 academic subjects from elementary mathematics to professional law and medicine) have been particularly influential. Both require knowledge integration and reasoning rather than simple pattern completion.

Performance on these benchmarks has tracked model scale in a roughly predictable way, with sharp discontinuities for in-context learning variants. Interestingly, some tasks within these benchmarks showed emergent behavior: performance was near random below a scale threshold and substantially above chance above it, while other tasks showed smooth scaling curves. Tasks that are emergent tend to require either a particularly rare capability (like understanding a mathematical concept that appears infrequently in training data) or the composition of multiple capabilities that each require significant scale individually.

The existence of these different scaling profiles within the same benchmark suggests that different tasks tap into qualitatively different capabilities, some of which require a certain minimal capacity to exhibit at all. This creates a challenge for benchmark design: a benchmark with a mix of smooth-scaling and emergent tasks will show an average performance curve that obscures both the emergent jumps and the smooth improvements, making it harder to understand what is happening at the capability level.

More recent benchmarks like GPQA (Graduate-Level Google-Proof Q&A), which consists of questions designed to be answerable by domain experts but not by web search, and FrontierMath, which consists of novel mathematical problems created after model training cutoffs, are attempting to address contamination and saturation concerns more systematically. These benchmarks show that even frontier models have substantial room for improvement, with accuracy rates well below expert human levels on the hardest questions.

Two domains deserve special attention as capability frontiers because they offer rigorous external verification: code and mathematics. Unlike natural language tasks where quality is often subjective, code either runs correctly or it does not, and mathematical proofs are either valid or invalid. This objectivity makes these domains particularly valuable for studying capabilities that transfer as opposed to fluent-sounding but potentially incorrect outputs.

The ability to write functional code from natural language descriptions has improved dramatically with scale. Modern large language models can produce working solutions to competitive programming problems, implement algorithms from high-level descriptions, debug code from error messages, identify security vulnerabilities, and translate programs across programming languages. This capability has practical importance, with language models now used routinely in software development workflows.

What makes code generation particularly interesting as a capability domain is the feedback loop it enables. A model that writes code and then executes it can observe whether the code works and use that feedback to iterate. This grounded evaluation loop is qualitatively different from generating natural language, where correctness is much harder to assess automatically. A model editing code based on test failures has a reliable signal; a model editing an essay based on vague feedback is working with a noisier signal.

Code synthesis benchmarks like HumanEval and MBPP have shown substantial improvement as models scale, but perhaps more telling is performance on harder benchmarks like competitive programming datasets from platforms like Codeforces or LeetCode. These require syntactically correct code and algorithms that handle edge cases efficiently, a capability that requires understanding the problem structure rather than retrieving common patterns. Performance on these harder benchmarks remains substantially below top human competitors, even for the largest models, suggesting that algorithmic creativity and deep problem analysis remain challenging.

The ability to work with code also enables interesting forms of mathematical and logical reasoning that go beyond direct proof. A model that can write and execute programs to check mathematical conjectures on small cases, or to search for counterexamples, is using computation as a cognitive tool in a way that can substantially exceed pure language-based reasoning.

Mathematical reasoning represents a frontier where language models show impressive but inconsistent performance. Models can solve multi-step algebra problems, apply calculus, reason about probability, and work through geometry proofs. On olympiad-level mathematics, performance has improved substantially with scale and with specialized training, with some models reaching performance levels competitive with strong human competitors on subsets of olympiad problems.

The interesting cases for understanding capability are where models fail. They tend to fail on problems that require a non-standard approach that differs from anything in the training distribution, problems requiring careful tracking of many interleaved constraints (where a single tracking error invalidates the entire solution), and problems where a common-sense shortcut would produce the wrong answer. These failure modes suggest that mathematical problem-solving involves at least two distinct components: skill-based (knowing which techniques to apply, recognizing problem types) and reasoning-based (working through the implications step by step), with models currently excelling more at the former than the latter.

This distinction is important. A student who knows calculus techniques can solve most standard calculus problems by pattern matching to technique types. A research mathematician solving a novel problem must reason from first principles, which requires something more than technique recognition. Current models appear to be strong pattern matchers but weaker at the kind of creative first-principles reasoning that characterizes original mathematical insight.

Formal theorem proving offers the cleanest evaluation because proofs can be mechanically verified. Systems that interact with proof assistants like Lean or Coq get immediate, reliable feedback on whether each step is valid. This setup has enabled interesting research: models trained with feedback from proof assistants substantially outperform those trained on informal mathematical text alone. The reason appears to be that the proof assistant provides a ground-truth signal at each step of the proof, allowing the model to learn which reasoning steps are valid without having to rely on self-evaluation or human feedback. This finding suggests that grounded verification loops are important for developing reliable mathematical reasoning, just as code execution is important for code generation.

Some recent work has explored "neurosymbolic" approaches that combine language model generation with formal symbolic reasoning: the language model generates proof strategies and candidate steps, while the proof assistant verifies them and signals when to backtrack. This division of labor plays to the strengths of each component: the language model is good at generating plausible-looking next steps from a large distribution of possibilities, while the symbolic system is reliable at verification. Early results have been impressive on certain classes of formal mathematics, though scaling to graduate-level research mathematics remains an open challenge.

Having surveyed emerging capabilities, it is important to be honest about their limits. The risk in this field is that impressive demonstrations create inflated expectations that distort research priorities and public understanding. The same models that solve olympiad-level mathematics problems can fail on elementary arithmetic when the problem is posed in an unusual way. The same models that follow complex multi-step instructions reliably can produce confidently stated factual errors on basic questions. Understanding this inconsistency is as important as understanding the capabilities.

Language models, even the most capable, are not reasoning systems in the way humans reason. They do not maintain persistent beliefs that update consistently across conversations, revise their world model based on new experience, or understand the consequences of their outputs in the world. Their impressive performance on reasoning benchmarks reflects, in part, the vast amount of human reasoning they have absorbed from training data. When asked to solve a problem similar to many they have seen, they perform well. When asked to solve a truly novel problem requiring original insight, performance degrades substantially.

This is not a criticism so much as a description. These systems are extraordinarily capable at pattern completion and interpolation within their training distribution. The boundary of their capability coincides roughly with the boundary of their training distribution, and that training distribution is vast. But the distribution is not infinite, and at its edges, capability drops off in ways that can be surprising and hard to predict in advance.

Reliability is a pervasive concern. A model that answers 95% of questions correctly in a domain is impressive by many standards, but for high-stakes decisions, 5% error rates can be unacceptable. Unlike traditional software with deterministic behavior, language model outputs vary with sampling temperature and can produce confident wrong answers. Building reliable systems from probabilistic components requires careful engineering: validation layers, verification tools, human oversight for critical decisions, and calibrated confidence signals that users can trust. The capability to give confident wrong answers is in some ways more dangerous than giving obviously wrong answers, because confident wrong answers may not be caught before they cause harm.

Consistency is another practical limitation. Models sometimes contradict themselves within a single conversation, taking opposite positions on the same question asked in different ways. This inconsistency reflects the statistical nature of generation: the model is sampling from a distribution, not consulting a fixed belief state. Ensuring consistent behavior across a conversation or across many queries to the same model requires additional mechanisms beyond what most current deployments provide. This is not a trivial engineering problem; it touches fundamental questions about what it means for a statistical system to "believe" something.

The gap between task-specific capability and general reasoning is real and important. Solving a math problem on a benchmark and being a reliable mathematical advisor are different things. The first requires matching a distribution; the second requires understanding when a method applies, recognizing when a problem is not well-formed, and knowing what you do not know. These meta-cognitive capabilities, knowing the boundaries of one's own knowledge and competence, are substantially less developed than object-level problem-solving. A model that does not know the limits of what it knows will confidently offer answers in domains where its knowledge is thin, without signaling the appropriate uncertainty to the user.

These limitations do not diminish the measured advances. They do, however, suggest a calibrated view: impressive on many tasks, unreliable on others, and requiring careful deployment strategies that account for failure modes. The most effective uses of language models pair them with verification tools, human oversight, and feedback loops that can catch and correct errors before they compound.

This section demonstrates several of the capability concepts discussed above through concrete implementation examples. We will explore in-context learning, chain-of-thought prompting, and tool-augmented reasoning using simulated environments so that the code runs without requiring API access.

In-context learning can be studied in miniature by examining how a model's behavior changes as more examples are added to the context. We will build a simple demonstration using a k-nearest-neighbor classifier that approximates the mechanics of in-context pattern matching. The key idea is that adding more examples to the context gives the learner more information about the task distribution, which reduces uncertainty and improves accuracy, exactly as adding more in-context examples helps a language model characterize the target task.

Chain-of-thought prompting improves performance by externalizing intermediate steps. We can demonstrate the core mechanism with a simple arithmetic reasoning task: computing the result of a multi-step word problem.

The key insight is that breaking a problem into steps reduces the complexity of each individual step. Even a simple calculator-style model improves when given intermediate results rather than being asked to compute everything at once. In the simulation below, direct solving accumulates noise proportional to problem length (simulating how errors compound when all computation must happen "in one go"), while chain-of-thought keeps each step's noise small (simulating how externalizing intermediate state prevents error accumulation).

The emergent capability phenomenon, where performance jumps at scale thresholds, can be visualized by modeling performance as a function of scale with a sigmoid transition curve. The model below represents two tasks that exhibit sharp emergence at different scale thresholds and one task that scales smoothly across all model sizes. This reflects the empirical finding that different capabilities have different scaling profiles, with some requiring a minimum capacity before they are exhibited at all.

The chart illustrates why emergent capabilities are surprising and difficult to predict. Below the threshold, more compute produces almost no benefit on the emergent tasks. Above it, a relatively small increase in scale produces dramatic improvement. For researchers and engineers planning compute budgets, this profile is challenging: you cannot see the capability coming before you have crossed the threshold. Tasks A and B also demonstrate that different capabilities emerge at different scales, so a model that exhibits one emergent capability may not yet have crossed the threshold for another.

Tool use is one of the most practically important capability frontiers. We can demonstrate the pattern with a simple agent loop that decides when to call an external function and integrates the result. The key design elements are: the agent embeds structured tool calls within its output, an executor dispatches those calls to the appropriate functions, and results are returned and integrated into the agent's running context.

The benchmark chart reveals an important asymmetry in current capability profiles: models score very high on instruction following, in-context learning, and classical reasoning benchmarks, but substantially lower on multimodal reasoning tasks that require integrating visual and linguistic information. This uneven profile reflects both the maturity of different training paradigms and the fundamental difficulty of tasks that require grounded perception. Language-only capabilities have benefited from far more training compute and data curation effort than multimodal capabilities, and the gap in benchmark performance roughly tracks this difference in research investment.

Capability frontiers in language AI are advancing on multiple dimensions simultaneously. The key takeaways from this chapter are:

• Emergent capabilities arise at scale thresholds and include in-context learning, chain-of-thought reasoning, and instruction following. These behaviors cannot be predicted by extrapolating from smaller models and represent qualitative shifts rather than gradual improvements. The mechanism behind emergence remains an active research question, with hypotheses ranging from implicit learning algorithms to threshold effects in learned statistical associations.

• World model evidence is mixed. Models encode substantial factual and relational knowledge, with evidence for structured internal representations including implicit spatial maps and entity state tracking. However, they struggle with compositional generalization and causal reasoning in ways that suggest their world knowledge remains shallow in important respects. The gap between appearing to know something and applying that knowledge reliably to novel situations is real and practically important.

• Planning and agency extend language models beyond text generation toward goal-directed behavior. Task decomposition, tool use, and self-correction are all achievable but degrade on long-horizon tasks requiring sustained working memory and reliable error recovery. The most reliable agentic behaviors are those grounded in external verification, such as code execution or symbolic proof checking.

• Multimodal reasoning brings vision and language together, enabling image understanding and visual question answering, but models still struggle with fine-grained spatial reasoning and are susceptible to hallucination driven by language priors overriding visual evidence. The gap between language-only and multimodal capabilities reflects both the difficulty of cross-modal integration and the relative maturity of language-only training.

• Honest assessment requires acknowledging that benchmark performance often overstates real capability due to pattern matching, data contamination, and the gap between task-specific performance and generalizable reasoning. Reliability, consistency, causal understanding, and reliable meta-cognition about the limits of one's knowledge remain open problems that constrain the deployment of even highly capable models in high-stakes applications.

The chapters that follow examine alignment challenges and societal implications, asking what these systems can do and what constraints and values we want to impose on how they do it.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about capability frontiers in language AI.