Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Emergence vs Metrics › Alternative Metrics Reveal Gradual Improvement

Michael BrenndoerferNovember 8, 202511 min read

Language AI HandbookAll chapters

Part 30 of 67Emergent Capabilities

Chapter 217Emergence vs Metrics › Alternative Metrics Reveal Gradual Improvement

Reading section 6 of 13 · Alternative Metrics Reveal Gradual Improvement · 11 minSection 6 of 13 · 11 min

Section 6 of 13 · 11 min read

Emergence vs Metrics › Alternative Metrics Reveal Gradual ImprovementOne section · Sections in this chapter6 / 13 ↓

Sections in this chapter13

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

If emergence is a metric artifact, using continuous metrics should reveal the underlying gradual improvement. This is exactly what researchers have found. The key insight is that continuous metrics preserve information about model confidence that binary metrics discard, letting us to observe the steady accumulation of capability that accuracy curves hide.

In practice, the choice of metric determines which story you see. A researcher using accuracy on 50-token tasks will observe dramatic emergence curves with necessary thresholds. A researcher using average log-probability on the same tasks will observe a boring, predictable sigmoid. Neither researcher is wrong about the data they report. They are measuring different things. One is measuring how often the model clears a difficult performance bar in a single attempt; the other is measuring how confidently the model "knows" the correct answer. The divergence between these two stories is the source of much confusion in the emergence debate.

Brier Score and Probability CalibrationLink Copied

The Brier score measures the mean squared error between predicted probabilities and actual outcomes. It was originally developed for weather forecasting, where simply predicting "rain" or "no rain" throws away valuable information about confidence levels. A forecast of 90% chance of rain is more useful than a forecast of 51% chance of rain, even though both might lead to the same recommendation: carry an umbrella.

\text{Brier} = \frac{1}{N} \sum_{i=1}^{N} (p_i - o_i)^2

where:

$\text{Brier}$ : the Brier score, measuring calibration quality (lower is better)
$N$ : the total number of predictions being evaluated
$p_i$ : the model's predicted probability for example $i$ (a value between 0 and 1, representing confidence in the positive class)
$o_i$ : the actual outcome for example $i$ (1 if the positive class occurred, 0 otherwise)
$(p_i - o_i)^2$ : the squared error for each prediction, penalizing confident wrong predictions heavily
$\frac{1}{N} \sum_{i=1}^{N}$ : the average over all $N$ predictions

The squaring is important: it penalizes large errors much more than small ones. A prediction of 0.9 for an event that does not happen ( $o_i = 0$ ) incurs error $(0.9 - 0)^2 = 0.81$ , while a prediction of 0.6 for the same outcome incurs only $(0.6 - 0)^2 = 0.36$ . This encourages the model to be conservative rather than overconfident. The quadratic penalty means that being confidently wrong is much worse than being uncertain, a property that aligns with our intuitions about what good predictions should look like.

The Brier score ranges from 0 (perfect predictions) to 1 (maximally wrong). A score of 0 would require perfect calibration: every event the model predicts with 70% confidence should occur 70% of the time, every event predicted with 30% confidence should occur 30% of the time, and so on. Unlike accuracy, which only cares whether $p_i > 0.5$ , Brier score rewards improvements in probability even when they don't change the final prediction. A model moving from 40% to 49% confidence on a correct answer improves its Brier score despite both predictions being "wrong" by the accuracy metric. Unlike accuracy, Brier score rewards probability improvements even when they don't cross the decision threshold.

This property makes Brier score useful for tracking capability development. When a model improves from 30% to 45% confidence in correct answers, it has learned something meaningful about the task. Accuracy is blind to this progress, but Brier score captures it. The model is becoming more reliable even before it crosses the threshold into "correct" territory.

Out[9]:

Visualization

Line plot comparing Brier score and accuracy metrics near the 0.5 decision threshold, showing accuracy's blind zone. — Comparison of Brier score and accuracy metrics near the 0.5 decision threshold. Brier score smoothly decreases as model confidence improves, while accuracy remains at zero until predictions cross the threshold, creating a blind zone where improvements are invisible.

Density plot showing probability distributions for low, medium, and high confidence models. — Probability density distributions for three models with different confidence levels in correct answers. Low confidence models show broad, uncertain distributions centered below 0.5, while high confidence models exhibit sharp peaks near 1.0.

In[10]:

Code

def brier_score(predicted_probs: np.ndarray, outcomes: np.ndarray) -> float:
    """Calculate Brier score (lower is better)."""
    return np.mean((predicted_probs - outcomes) ** 2)


def accuracy(predicted_probs: np.ndarray, threshold: float = 0.5) -> float:
    """Calculate accuracy using probability threshold."""
    predictions = predicted_probs >= threshold
    return np.mean(predictions)


# Simulate improving model across scale
np.random.seed(42)
n_examples = 1000
scales = np.linspace(18, 26, 9)

accuracy_by_scale = []
brier_by_scale = []

for scale in scales:
    # Underlying capability improves with scale
    base_capability = 0.3 + 0.6 / (1 + np.exp(-(scale - 22)))

    # Generate predicted probabilities with noise
    probs = np.clip(
        base_capability + np.random.normal(0, 0.15, n_examples), 0.01, 0.99
    )

    # True outcomes: 1 if correct answer should be chosen
    # Higher capability means higher probability of correct answer
    outcomes = (np.random.random(n_examples) < probs).astype(float)

    accuracy_by_scale.append(accuracy(probs))
    brier_by_scale.append(brier_score(probs, outcomes))

# Store final values for reporting
final_accuracy = accuracy_by_scale[-1]
final_brier = brier_by_scale[-1]
initial_accuracy = accuracy_by_scale[0]
initial_brier = brier_by_scale[0]

Out[11]:

Visualization

Dual-axis plot comparing smooth Brier score improvement to step-like accuracy curve. — Dual-axis plot comparing continuous and discontinuous metrics measuring the same underlying capability. Brier score (lower is better) shows smooth, gradual improvement across all training scales, while accuracy exhibits a sharp S-curve transition around 10^22 FLOPs, creating the illusion of sudden emergence.

Out[12]:

Console

Accuracy improvement: 10.6% -> 99.4%
Brier score improvement: 0.192 -> 0.115

The Brier score improves smoothly across all scales, revealing the gradual capability development that accuracy obscures. While accuracy jumps from near-random to near-perfect performance, the Brier score shows steady, predictable improvement throughout. This is the same model, learning in the same way. Only our measurement lens has changed. The Brier score tells a story of continuous progress, while accuracy tells a story of sudden breakthrough. One of these stories is an artifact of the metric; the other reflects the underlying reality.

Token-Level Log-ProbabilitiesLink Copied

The most direct continuous metric is the average log-probability assigned to correct tokens. As we covered in our discussion of perplexity in Part II, this measures how surprised the model is by the correct answer. A model that assigns high probability to correct tokens is less surprised when it sees them, showing it has learned something about the structure of correct responses.

The relationship between log-probability and perplexity is exact: perplexity is $e$ raised to the negative average log-probability. When we discussed perplexity as a measure of language model quality, we were really discussing average log-probability in disguise. Perplexity is useful for measuring overall language modeling quality on held-out text, but average log-probability on answer tokens measures something more specific: how well the model has internalized the knowledge needed to answer questions correctly. This distinction matters because a model could have excellent perplexity on general text while still being uncertain about specific factual questions. Tracking log-probability on task-specific answer tokens gives a more targeted signal about capability development for those particular tasks.

For a correct output sequence $y_1, y_2, \ldots, y_n$ , the average log-probability is:

\text{AvgLogProb} = \frac{1}{n} \sum_{i=1}^{n} \log p(y_i | y_{<i}, x)

where:

$\text{AvgLogProb}$ : the average log-probability score, also known as the negative of per-token cross-entropy loss (higher/less negative indicates better predictions, with 0 being the theoretical maximum for perfect confidence)
$n$ : the number of tokens in the correct output sequence
$y_i$ : the $i$ -th token in the correct output sequence
$y_{<i}$ : all tokens preceding position $i$ (the context so far)
$x$ : the input prompt or question
$p(y_i | y_{<i}, x)$ : the model's predicted probability for token $y_i$ given the input and all previous correct tokens
$\log$ : the natural logarithm, which converts probabilities (0 to 1) to negative numbers (higher/less negative = more confident)
$\frac{1}{n} \sum_{i=1}^{n}$ : the average over all $n$ tokens in the sequence

The logarithm serves a specific purpose here: it converts multiplicative relationships into additive ones. Recall that sequence probability is the product of token probabilities: $P(\text{seq}) = \prod_i p_i$ . Taking logs turns this into a sum: $\log P(\text{seq}) = \sum_i \log p_i$ . This makes the metric interpretable as an average per-token score rather than a product that shrinks toward zero for long sequences. By working in log space, we avoid the amplification problem that plagues exact-match metrics. Long sequences don't artificially suppress the measured capability.

This metric captures how "surprised" the model is by the correct answer. A model assigning 90% probability to each correct token will have a higher (less negative) average log-probability than one assigning 60%, even if both models ultimately generate the correct sequence.

To make this concrete: $\log(0.9) \approx -0.105$ while $\log(0.6) \approx -0.511$ . The 90%-confident model scores nearly 5 times better per token on this metric, capturing the substantial difference in capability that accuracy would miss if both models cross the decision threshold. This 5x difference in scores reflects a meaningful difference in the models' internal states. One model has learned something meaningful about what makes answers correct, while the other is still somewhat uncertain. The average log-probability preserves this information rather than discarding it.

Out[13]:

Visualization

Line plot showing smooth, gradual improvement in average log-probability per token as compute increases. — Average log-probability per token showing smooth, continuous improvement as training compute increases. The metric captures gradual capability development without the threshold effects that create apparent discontinuities.

Line plot showing sharp S-curve transition in exact match accuracy around 10^22 FLOPs. — Exact-match accuracy for 20-token sequences, exhibiting a sharp S-curve transition around 10^22 FLOPs. Despite identical underlying capability improvements, the discontinuous metric creates the illusion of sudden emergence.

Studies examining emergence have found that this metric improves log-linearly with compute even for tasks where accuracy shows sharp transitions. The model's uncertainty about the correct answer decreases gradually; it's only the binary accuracy measurement that creates the appearance of sudden capability.