LLM Emergence: Are Capabilities Real or Metric Artifacts?

The previous chapters painted a striking picture of emergent capabilities: abilities that appear suddenly at specific scales and are absent in smaller models but present in larger ones. These discontinuous jumps seemed to suggest that scaling could change how large language models acquire new capabilities. But a necessary question has emerged in recent research: are we observing real phase transitions in model capability, or artifacts of how we measure performance?

This distinction matters enormously. If emergence is real, it suggests that scaling models will produce entirely new capabilities in unpredictable ways, making future AI development fundamentally hard to anticipate. If emergence is primarily a measurement artifact, capabilities may develop more gradually than accuracy curves suggest, and we can better predict what larger models will achieve. The second interpretation also changes how we should allocate research resources: rather than preparing for surprising capability jumps, we can focus on the steady engineering work of improving per-token prediction across scale.

The stakes extend beyond academic debate. Research labs, safety teams, and policymakers use emergence claims to justify urgent timelines and policy responses. If a model could unexpectedly "wake up" to new dangerous capabilities at some necessary scale threshold, that demands one kind of response. If capabilities develop predictably and smoothly, that demands a different, more measured approach. Getting this right matters scientifically and practically.

The answer, as we'll see, lies in understanding the mathematics of measurement itself. Before we can ask whether capabilities emerge discontinuously, we must ask a more basic question: what exactly are our benchmarks measuring?

When we evaluate language models on tasks, we face a basic choice: how do we convert continuous model outputs into discrete success/failure judgments? This choice has measurable implications for how capabilities appear to develop.

Think of a language model as a machine that assigns probabilities to every token in its vocabulary at each position in a sequence. When answering a multiple-choice question, the model doesn't internally choose "A." It assigns probability mass across all possible continuations, and we read off which choice received the most probability. The model's internal state contains a rich distribution of confidence, but the moment we ask "did it get the question right?" we collapse that entire distribution into a single bit of information: 0 or 1, wrong or right.

Consider a multiple-choice question with four options. A model produces probability distributions over tokens, which we interpret as confidence in each answer choice. But benchmarks don't report these probabilities directly; they report whether the model got the answer right or wrong. This binary reduction throws away most of the information about what the model learned.

To see how severe this information loss is, consider a student taking a test. One student confidently writes the correct answer with complete certainty. Another student narrows down to two choices and makes an educated guess that happens to be right. A third student was 90% sure of the correct answer but second-guessed themselves at the last moment. Traditional grading treats all three identically: all three receive full marks. Yet clearly these students have different levels of understanding. The same information collapse occurs when we evaluate language models using binary accuracy metrics.

The key insight is that this information loss is not uniformly distributed across scale. Smaller models tend to hover near the decision boundary, where accuracy discards the most information. As models improve through scaling, they push their probabilities progressively farther from the boundary, but accuracy only "notices" this improvement once the boundary is crossed. This asymmetry means that accuracy metrics systematically understate early capability gains and then suddenly "catch up" when probabilities finally exceed 50%, creating the illusion of discontinuous improvement where none exists.

The metrics we use for evaluation fall into two broad categories. Continuous metrics like cross-entropy loss, Brier score, or token-level log-probabilities can reveal gradual improvements. Discontinuous metrics like accuracy, exact match, or pass@1 collapse this information into binary outcomes.

To understand the difference concretely: if a model improves its confidence in the correct answer from 30% to 49%, a continuous metric like Brier score captures this substantial improvement, while accuracy still reports zero, since the model was "wrong" both times. The model has learned something valuable, becoming nearly twice as confident in the correct answer, yet our measurement instrument is blind to this progress.

Most emergence papers relied on discontinuous metrics. The sudden appearance of capabilities they documented may tell us more about the metrics than about the models.

To understand why discontinuous metrics create apparent emergence, we examine the mathematics of threshold effects. The core insight is that when we demand perfect sequential correctness, we transform what might be gradual improvement into an all-or-nothing proposition.

Consider a task where a model must generate an exact string like "The answer is 42." The model produces this string autoregressively, with some probability $p_i$ of generating the correct token at position $i$ given all previous tokens were generated correctly. Each token represents a checkpoint that the model must pass. Failing at any point means failing entirely.

The probability of generating the entire correct sequence is:

where:

• $P(\text{correct sequence})$: the probability of generating the entire target string correctly
• $p_i$: the probability of generating the $i$-th correct token, given all previous tokens were correct
• $n$: the total number of tokens in the target sequence
• $\prod_{i=1}^{n}$: the product operator over all $n$ tokens, applying the chain rule of probability for sequential generation, where each token's probability is conditioned on all previous tokens being correct

This formula shows that sequential correctness is fragile. Unlike addition, where errors might partially cancel, multiplying probabilities means every imperfect step compounds the risk of failure. A chain is only as strong as its weakest link, and here we're multiplying all the links together. Think of it like a relay race where each runner must complete their leg perfectly. One stumble anywhere and the entire team loses, regardless of how well everyone else performed.

This multiplicative relationship is important: even small imperfections compound. If each token has a 95% chance of being correct, the probability of getting all tokens right shrinks rapidly as the sequence grows longer. What seems like excellent per-step performance (after all, 95% accuracy sounds impressive) leads to surprisingly poor overall outcomes when steps must chain together.

If we assume roughly uniform per-token accuracy $p$ across all $n$ tokens, this becomes:

This exponential relationship creates a threshold effect. The exponent $n$ acts as an amplifier, transforming modest changes in $p$ into dramatic changes in overall success probability. Suppose the model's per-token accuracy improves linearly with log compute from 80% to 99% over several orders of magnitude of scaling. For a 10-token answer:

The per-token accuracy improved smoothly from 80% to 99%, but sequence accuracy appears to "emerge" around the 95% per-token threshold. A model with 80% per-token accuracy looks like it has nearly zero capability on this task, but it is closer to succeeding than the metric suggests. This table shows a clear asymmetry: the jump from 95% to 99% per-token accuracy (a mere 4 percentage points) produces roughly the same sequence accuracy improvement as the entire journey from 80% to 95% (15 percentage points). The mathematics of multiplication creates this hockey-stick pattern automatically.

The heatmap reveals the "emergence zone" where sequence accuracy transitions rapidly from near-zero (red) to high performance (green). Notice how this transition zone shifts rightward for longer sequences. A 50-token task requires over 95% per-token accuracy to achieve even 50% sequence accuracy, while a 10-token task reaches the same performance at around 93% per-token accuracy.

Longer sequences amplify this effect dramatically, turning modest measurement issues into severe distortions of apparent capability. The mathematical relationship between sequence length and success probability follows an inexorable exponential decay. For a 50-token sequence:

where:

• $0.95$: the per-token accuracy (95% chance of getting each token correct)
• $50$: the number of tokens in the target sequence
• $0.95^{50}$: the result of multiplying 0.95 by itself 50 times
• $\approx 0.077$: approximately 7.7% probability of generating the entire sequence correctly

To see why this happens, consider that $0.95^{50}$ means multiplying 0.95 by itself 50 times. Each multiplication slightly shrinks the result: $0.95 \times 0.95 = 0.9025$, then $0.9025 \times 0.95 = 0.857$, and so on. After 50 such multiplications, even starting from 95% per step, the cumulative probability has decayed to under 8%. This decay is relentless and mechanical. No amount of hoping or clever prompting can overcome the basic mathematics of multiplied probabilities.

Even a 95% per-token model achieves less than 8% exact match on a 50-token sequence. This model has substantial capability, but the exact match metric makes it appear completely incompetent. Consider a surgeon who performs each step of a 50-step procedure with 95% success. We would consider them highly skilled. Yet our evaluation framework would label their overall performance as a failure more than 92% of the time.

The relationship between per-token probability and sequence accuracy follows the exponential formula $P = p^n$, where small changes in $p$ produce dramatic changes in $P$ for large $n$. This mathematical structure is not a bug in our analysis; it is an accurate description of how exact-match evaluation works. The question is whether this evaluation accurately reflects the capability we care about, or whether it creates artificial cliffs that obscure underlying progress.

Notice how longer sequences push the "emergence threshold" higher. A task requiring 50-token outputs won't show above-chance accuracy until per-token probability exceeds roughly 97%. This creates the illusion that long-form generation capabilities emerge suddenly at large scales. The capability didn't emerge suddenly. Our measurement tool simply wasn't sensitive enough to detect the gradual improvement happening underneath.

Let's create a simulation that demonstrates how smooth underlying improvements can appear as emergent transitions when measured with accuracy. This visualization will make the abstract mathematics concrete by showing exactly how the same underlying capability trajectory produces different apparent patterns depending on our choice of metric.

The left panel shows smooth, gradual improvement in the model's underlying capability. The right panel shows how this same improvement appears when measured by accuracy on sequences of different lengths. The 50-token task appears to "emerge" suddenly around $10^{24}$ FLOPs, while the 5-token task shows more gradual improvement. Yet the underlying capability change is identical. This visualization shows the core argument: emergence can be an optical illusion created by our measurement choices, not a basic property of how capabilities develop.

The same mathematics applies to multiple-choice questions, though the mechanism is slightly different. For a four-choice question, random performance is 25%. If the model must distinguish between answers that differ in the first few tokens, effective "sequence length" is short, and improvements appear more gradual.

However, many benchmark questions require understanding long contexts or performing multi-step reasoning before selecting an answer. The model's internal representations must correctly process many tokens to arrive at the right answer, even if the output is a single letter. This introduces a subtlety that the simple sequence-length formula does not capture directly, though the underlying multiplicative probability dynamics still apply. The difficulty of a task depends on the number of output tokens we demand and the number of internal computational steps the model must execute correctly.

This explains why complex reasoning tasks like those in BIG-Bench show more pronounced emergence than simple classification tasks: they have longer effective reasoning lengths, which amplifies the threshold effect. A task requiring the model to parse a complex sentence, identify relevant entities, retrieve world knowledge, apply logical rules, and synthesize a conclusion may involve the equivalent of a 30 or 40 "step" reasoning chain internally. Even if the final output is just "A" or "B", the threshold mathematics of multiplicative probabilities still applies to those internal steps, creating the same artificial emergence patterns we observe in long-form generation tasks.

If emergence is a metric artifact, using continuous metrics should reveal the underlying gradual improvement. This is exactly what researchers have found. The key insight is that continuous metrics preserve information about model confidence that binary metrics discard, letting us to observe the steady accumulation of capability that accuracy curves hide.

In practice, the choice of metric determines which story you see. A researcher using accuracy on 50-token tasks will observe dramatic emergence curves with necessary thresholds. A researcher using average log-probability on the same tasks will observe a boring, predictable sigmoid. Neither researcher is wrong about the data they report. They are measuring different things. One is measuring how often the model clears a difficult performance bar in a single attempt; the other is measuring how confidently the model "knows" the correct answer. The divergence between these two stories is the source of much confusion in the emergence debate.

The Brier score measures the mean squared error between predicted probabilities and actual outcomes. It was originally developed for weather forecasting, where simply predicting "rain" or "no rain" throws away valuable information about confidence levels. A forecast of 90% chance of rain is more useful than a forecast of 51% chance of rain, even though both might lead to the same recommendation: carry an umbrella.

where:

• $\text{Brier}$: the Brier score, measuring calibration quality (lower is better)
• $N$: the total number of predictions being evaluated
• $p_i$: the model's predicted probability for example $i$ (a value between 0 and 1, representing confidence in the positive class)
• $o_i$: the actual outcome for example $i$ (1 if the positive class occurred, 0 otherwise)
• $(p_i - o_i)^2$: the squared error for each prediction, penalizing confident wrong predictions heavily
• $\frac{1}{N} \sum_{i=1}^{N}$: the average over all $N$ predictions

The squaring is important: it penalizes large errors much more than small ones. A prediction of 0.9 for an event that does not happen ($o_i = 0$) incurs error $(0.9 - 0)^2 = 0.81$, while a prediction of 0.6 for the same outcome incurs only $(0.6 - 0)^2 = 0.36$. This encourages the model to be conservative rather than overconfident. The quadratic penalty means that being confidently wrong is much worse than being uncertain, a property that aligns with our intuitions about what good predictions should look like.

The Brier score ranges from 0 (perfect predictions) to 1 (maximally wrong). A score of 0 would require perfect calibration: every event the model predicts with 70% confidence should occur 70% of the time, every event predicted with 30% confidence should occur 30% of the time, and so on. Unlike accuracy, which only cares whether $p_i > 0.5$, Brier score rewards improvements in probability even when they don't change the final prediction. A model moving from 40% to 49% confidence on a correct answer improves its Brier score despite both predictions being "wrong" by the accuracy metric. Unlike accuracy, Brier score rewards probability improvements even when they don't cross the decision threshold.

This property makes Brier score useful for tracking capability development. When a model improves from 30% to 45% confidence in correct answers, it has learned something meaningful about the task. Accuracy is blind to this progress, but Brier score captures it. The model is becoming more reliable even before it crosses the threshold into "correct" territory.

The Brier score improves smoothly across all scales, revealing the gradual capability development that accuracy obscures. While accuracy jumps from near-random to near-perfect performance, the Brier score shows steady, predictable improvement throughout. This is the same model, learning in the same way. Only our measurement lens has changed. The Brier score tells a story of continuous progress, while accuracy tells a story of sudden breakthrough. One of these stories is an artifact of the metric; the other reflects the underlying reality.

The most direct continuous metric is the average log-probability assigned to correct tokens. As we covered in our discussion of perplexity in Part II, this measures how surprised the model is by the correct answer. A model that assigns high probability to correct tokens is less surprised when it sees them, showing it has learned something about the structure of correct responses.

The relationship between log-probability and perplexity is exact: perplexity is $e$ raised to the negative average log-probability. When we discussed perplexity as a measure of language model quality, we were really discussing average log-probability in disguise. Perplexity is useful for measuring overall language modeling quality on held-out text, but average log-probability on answer tokens measures something more specific: how well the model has internalized the knowledge needed to answer questions correctly. This distinction matters because a model could have excellent perplexity on general text while still being uncertain about specific factual questions. Tracking log-probability on task-specific answer tokens gives a more targeted signal about capability development for those particular tasks.

For a correct output sequence $y_1, y_2, \ldots, y_n$, the average log-probability is:

where:

• $\text{AvgLogProb}$: the average log-probability score, also known as the negative of per-token cross-entropy loss (higher/less negative indicates better predictions, with 0 being the theoretical maximum for perfect confidence)
• $n$: the number of tokens in the correct output sequence
• $y_i$: the $i$-th token in the correct output sequence
• $y_{<i}$: all tokens preceding position $i$ (the context so far)
• $x$: the input prompt or question
• $p(y_i | y_{<i}, x)$: the model's predicted probability for token $y_i$ given the input and all previous correct tokens
• $\log$: the natural logarithm, which converts probabilities (0 to 1) to negative numbers (higher/less negative = more confident)
• $\frac{1}{n} \sum_{i=1}^{n}$: the average over all $n$ tokens in the sequence

The logarithm serves a specific purpose here: it converts multiplicative relationships into additive ones. Recall that sequence probability is the product of token probabilities: $P(\text{seq}) = \prod_i p_i$. Taking logs turns this into a sum: $\log P(\text{seq}) = \sum_i \log p_i$. This makes the metric interpretable as an average per-token score rather than a product that shrinks toward zero for long sequences. By working in log space, we avoid the amplification problem that plagues exact-match metrics. Long sequences don't artificially suppress the measured capability.

This metric captures how "surprised" the model is by the correct answer. A model assigning 90% probability to each correct token will have a higher (less negative) average log-probability than one assigning 60%, even if both models ultimately generate the correct sequence.

To make this concrete: $\log(0.9) \approx -0.105$ while $\log(0.6) \approx -0.511$. The 90%-confident model scores nearly 5 times better per token on this metric, capturing the substantial difference in capability that accuracy would miss if both models cross the decision threshold. This 5x difference in scores reflects a meaningful difference in the models' internal states. One model has learned something meaningful about what makes answers correct, while the other is still somewhat uncertain. The average log-probability preserves this information rather than discarding it.

Studies examining emergence have found that this metric improves log-linearly with compute even for tasks where accuracy shows sharp transitions. The model's uncertainty about the correct answer decreases gradually; it's only the binary accuracy measurement that creates the appearance of sudden capability.

In 2023, researchers at Stanford and Google published a provocative paper titled "Are Emergent Abilities of Large Language Models a Mirage?" They systematically tested whether apparent emergence could be explained by metric choice alone. The paper made an unusually direct argument: the researchers showed that a researcher can create the appearance of emergence in any dataset simply by choosing an appropriate metric, and can make emergence disappear by choosing a different metric. If emergence were a real phenomenon, switching metrics should not cause it to vanish entirely.

The researchers took tasks previously claimed to show emergence and re-evaluated models using continuous metrics. They found that for nearly all tasks examined:

1. Accuracy showed sharp transitions consistent with prior emergence claims
2. Continuous metrics showed smooth, predictable improvement across all scales
3. The transition point was predictable from the metric and task structure

This pattern held across arithmetic, translation, word unscrambling, and other tasks that had been cited as evidence for emergent capabilities. The researchers also showed the complementary result: if you take a dataset where capabilities appear to develop smoothly, you can manufacture apparent emergence by switching to a harder metric that requires near-perfect token-level accuracy. The "emergence" appears or disappears based on measurement choices, not based on anything the model does differently.

The experimental design was careful to use the same model weights and the same tasks throughout. Nothing about the model changed between the "emergence" and "no emergence" conditions. Only the metric changed. This is the strongest possible evidence for the measurement artifact hypothesis.

The mirage paper made an important distinction between two types of emergence:

Metric-induced emergence occurs when smooth underlying improvements appear discontinuous due to threshold effects in measurement. This is not truly "emergent" in any meaningful sense; the capability develops gradually and predictably. The transition point can be calculated in advance from the metric structure and the expected improvement trajectory.

True emergence would involve discontinuous changes in underlying capability: qualitative changes in how the model processes information that couldn't be predicted from smaller-scale behavior. True emergence would persist regardless of which metric you use to measure it, because the discontinuity would be in the underlying model behavior rather than in the measurement lens.

The paper argued that most documented cases of emergence are metric-induced, not true emergence. This does not mean the capabilities are unimportant or that scaling does not produce remarkable abilities. It means the discontinuity is in our measurement, not in the model's developmental trajectory. The capabilities were developing all along; we simply couldn't see them with the tools we were using.

The metric critique doesn't eliminate the possibility of true emergence. Several mechanisms could produce real discontinuities, and it is worth understanding each carefully so you can evaluate specific capability claims with the appropriate skepticism rather than blanket dismissal.

The key test for any emergence claim is: does the apparent discontinuity survive a switch to a continuous metric? If yes, you may be seeing real emergence. If no, you have a measurement artifact. In practice, this test is often not run, because it requires access to the model's raw probability outputs rather than just its final answers. Many commercial model evaluations report only accuracy, which makes it impossible to retroactively apply the continuous metric test.

Neural networks may undergo qualitative changes in internal representations at certain scales. For example, a model might transition from storing specific facts to learning generalizable rules. Such transitions could produce real capability discontinuities that appear in token-level probabilities, not just in binary accuracy scores.

The mechanistic interpretability research we touched on in earlier chapters provides one window into this question. If researchers can identify circuits that implement specific capabilities, and if those circuits appear abruptly at a threshold scale rather than growing gradually, that would be strong evidence for representational emergence. The challenge is that interpretability at scale is difficult, and we don't yet have reliable tools to trace the formation of specific circuits across many different model sizes.

As we discussed in our chapter on in-context learning emergence, some capabilities may require specific computational circuits that only form reliably at sufficient scale. The grokking phenomenon, which we'll explore in an upcoming chapter, shows that models can suddenly shift from memorization to generalization after extended training.

Some capabilities require combining multiple subskills. If each sub-skill develops gradually but the combined capability requires all components simultaneously, the composite capability could emerge suddenly even with smooth sub-skill development. This mechanism differs fundamentally from metric-induced emergence. The discontinuity arises from the logical structure of the task itself, not from how we measure performance.

To understand this mechanism, consider learning to drive a car. Each individual skill (steering, braking, checking mirrors, reading traffic signs) can be practiced and improved gradually. But safely driving through an intersection requires all of these skills operating together correctly. A driver who excels at steering but forgets to check mirrors will fail the composite task. Similarly, one who reads signs but brakes too late will also fail. The intersection-driving capability might "emerge" suddenly once all component skills exceed their required thresholds, even though no individual skill shows a phase transition.

This type of compositional emergence is more subtle than metric-induced emergence. The individual skills develop smoothly, but the task requiring all skills shows a sharp transition. This could represent true emergence, or it could be another form of measurement artifact depending on how we define "capability." The key distinction is that compositional emergence reflects something real about task structure. Some tasks require multiple competencies to work together, rather than being purely an artifact of how we measure success.

Understanding the distinction between metric-induced and true emergence affects how we evaluate and study language models. The implications are concrete and actionable: the benchmarks you design, the metrics you report, and the conclusions you draw from scaling experiments all depend on getting this right.

Consider a practical scenario: you are a researcher designing an evaluation to test whether a new model architecture has better reasoning capabilities than the baseline. You run both models on a math reasoning benchmark and report accuracy. You observe that the new architecture shows "emergence" at a certain scale where the baseline does not. Is this evidence that your architecture is qualitatively better? Not necessarily. It might simply mean that your architecture has slightly better per-token accuracy on reasoning steps, and the combination of that small per-step advantage with the multiplicative dynamics of exact-match evaluation makes the gap look much larger than it is. Reporting both Brier score and accuracy would let you distinguish these cases.

When studying capability development across scales, several practices can help distinguish metric artifacts from real phenomena:

1. Report multiple metrics. Always include at least one continuous metric alongside accuracy. Log-probability on correct tokens, Brier score, or calibration curves reveal whether underlying capability is improving.

2. Analyze per-token probabilities. For generation tasks, examine probability trajectories token-by-token rather than only sequence-level success.

3. Consider task decomposition. If a task can be broken into sub-tasks, measure performance on components separately to identify which, if any, show real discontinuities.

4. Control for effective sequence length. Tasks with longer required outputs or reasoning chains will show sharper transitions mechanically. Compare tasks of similar effective length when claiming differential emergence.

5. Test metric predictions. If emergence is metric-induced, changing the metric should eliminate the discontinuity. Test this explicitly.

The metric perspective helps with capability prediction. If capabilities develop smoothly, we can extrapolate performance to larger scales more reliably. The Chinchilla scaling laws we studied in Part XXIII become more applicable. Rather than expecting unpredictable capability jumps, we can forecast how much compute is needed for specific capability levels. This is enormously useful for AI development planning: if you know that a capability follows a smooth sigmoid in log-probability space, you can estimate with reasonable confidence how much additional compute you need to reach a target performance level.

In practice, this means that per-token log-probability on task-relevant tokens often provides better predictions of future capability than accuracy does. A researcher who tracks log-probability improvements can see capability growth that an accuracy-focused researcher completely misses. Over time, this translates into more accurate forecasts of model behavior at the next scale.

However, this optimistic view assumes metric-induced emergence explains most cases. If some capabilities emerge through circuit formation or representational phase transitions, prediction remains challenging for those specific abilities. The current evidence suggests that metric-induced emergence is far more common than true emergence, but the research is not definitive. Treating all emergence claims as measurement artifacts until proven otherwise is a reasonable default posture for a careful researcher.

The metric-induced emergence hypothesis explains many cases, but it is not universally accepted, and important questions remain. Understanding these limitations will help you engage critically with the ongoing literature rather than treating the mirage paper as the final word.

One limitation is that the analysis focuses on task-level accuracy versus token-level probability. Some researchers argue this misses the possibility of emergence in internal representations. A model might develop new computational circuits or representational structures discontinuously, even if this manifests as smooth token-probability improvements. Studying internal representations directly, through probing or mechanistic interpretability techniques, could reveal forms of emergence invisible to behavioral metrics. Activation patching, causal tracing, and circuit analysis tools have found evidence for specific computational mechanisms implementing particular capabilities, and in principle these mechanisms could appear abruptly rather than gradually.

Additionally, the mirage paper primarily examined relatively simple tasks. More complex capabilities, like multi-step reasoning, theory of mind, or creative problem-solving, might behave differently. These capabilities may require qualitative changes in processing that emerge rather than develop gradually. The chain-of-thought reasoning we explored in the previous chapter could involve circuit formation that enables new types of computation. It is entirely possible that simple capabilities follow the measurement-artifact story while complex, multi-step capabilities involve real representational transitions. We do not yet have enough evidence to settle this question.

There is also debate about whether this distinction matters practically. If a capability appears to emerge regardless of the underlying mechanism, users experience a discontinuous jump in model utility. From an engineering perspective, metric-induced emergence may be indistinguishable from true emergence in its practical effects. A practitioner who is trying to deploy models for specific tasks does not necessarily care whether the improved performance comes from a phase transition or from gradual improvement that finally cleared a threshold. What matters is that the model now works for the application.

The metric perspective also does not explain why different capabilities emerge at different scales. Even if all emergence is metric-induced, understanding which capabilities require more scale and why remains an important research question. Arithmetic tasks tend to "emerge" earlier than abstract reasoning tasks, and this ordering is not fully explained by sequence length alone. The structure of the task, the frequency of relevant patterns in training data, and the computational complexity of the required operations all likely play roles.

Finally, one subtle point deserves attention. The mirage paper and similar critiques focus on what we can measure from the outside. A model is a black box in this framing: we feed it inputs and observe its outputs. But internal emergence, where the model's representations or computations change qualitatively even without obvious external signals, cannot be ruled out by behavioral evidence alone. The absence of behavioral emergence does not prove the absence of representational emergence; it only proves that any representational changes happening internally are not yet large enough to push token probabilities across measurement thresholds. As interpretability tools mature, this question may become more tractable.

This chapter examined the relationship between how we measure capabilities and how capabilities appear to develop. The central question was whether emergence is real or whether it is a feature of our measurement tools rather than the models themselves.

The mathematical foundation is straightforward but important. Exact-match accuracy applies a multiplicative structure to sequential tasks: every token must be correct, and failing any single token means failing the entire task. This creates an exponential relationship between per-token accuracy and sequence accuracy, which mechanically produces sharp threshold effects as models improve. The same gradual improvement in per-token probability that looks smooth in isolation becomes a dramatic S-curve when raised to the power of the sequence length.

The empirical evidence supports the measurement-artifact interpretation for most documented cases of emergence. When researchers switched from accuracy to continuous metrics like Brier score or average log-probability, the apparent discontinuities vanished. The capabilities were developing all along; accuracy was just too coarse an instrument to detect them.

The key insights include:

• Discontinuous metrics create apparent emergence. Accuracy and exact match metrics can make smooth underlying improvements appear as sudden capability jumps due to threshold effects.

• Sequence length amplifies discontinuity. Tasks requiring longer outputs or more reasoning steps show sharper apparent transitions, even when per-token capability improves smoothly.

• Continuous metrics reveal gradual improvement. Brier score, log-probabilities, and other continuous metrics typically show smooth, predictable scaling where accuracy shows sharp emergence.

• Most documented emergence may be metric-induced. Research re-examining emergence claims found that continuous metrics eliminated apparent discontinuities for most tasks studied.

• True emergence remains possible. Compositional requirements, circuit formation, and representational phase transitions could produce real capability discontinuities, though these are harder to document conclusively.

• Evaluation practices should adapt. Researchers should report multiple metrics and analyze per-token probabilities to distinguish metric artifacts from real phenomena.

The broader lesson is one about epistemic humility in empirical science. The tools we use to observe a system inevitably shape what we see. Language model capabilities are no exception. A researcher who uses only accuracy to study capability development will see emergence everywhere. A researcher who uses continuous metrics will see smooth, predictable growth. The truth is probably somewhere in between: mostly smooth growth with occasional real discontinuities that require harder-to-detect internal analysis to identify. Being clear about which type of evidence you are reporting, and what it does and does not prove, is the mark of careful scientific practice.

The key parameters and concepts for understanding emergence metrics are:

• per_token_prob (p): The probability of generating each correct token. Small improvements in per-token probability can produce dramatic changes in sequence accuracy.
• sequence_length (n): The number of tokens in the target output. Longer sequences amplify threshold effects exponentially.
• Brier score: A continuous metric measuring mean squared error between predictions and outcomes. Unlike accuracy, it captures improvement even below decision thresholds.
• threshold: The decision boundary (typically 0.5) where predictions flip from negative to positive class. Accuracy only changes when probabilities cross this boundary.

The next chapter examines inverse scaling, a counterintuitive phenomenon where larger models sometimes perform worse on specific tasks. This challenges even the basic assumption that more scale produces more capability.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about emergence and measurement artifacts.