Emergence vs Metrics › Emergence vs Metrics

The previous chapters painted a striking picture of emergent capabilities: abilities that appear suddenly at specific scales and are absent in smaller models but present in larger ones. These discontinuous jumps seemed to suggest that scaling could change how large language models acquire new capabilities. But a necessary question has emerged in recent research: are we observing real phase transitions in model capability, or artifacts of how we measure performance?

This distinction matters enormously. If emergence is real, it suggests that scaling models will produce entirely new capabilities in unpredictable ways, making future AI development fundamentally hard to anticipate. If emergence is primarily a measurement artifact, capabilities may develop more gradually than accuracy curves suggest, and we can better predict what larger models will achieve. The second interpretation also changes how we should allocate research resources: rather than preparing for surprising capability jumps, we can focus on the steady engineering work of improving per-token prediction across scale.

The stakes extend beyond academic debate. Research labs, safety teams, and policymakers use emergence claims to justify urgent timelines and policy responses. If a model could unexpectedly "wake up" to new dangerous capabilities at some necessary scale threshold, that demands one kind of response. If capabilities develop predictably and smoothly, that demands a different, more measured approach. Getting this right matters scientifically and practically.

The answer, as we'll see, lies in understanding the mathematics of measurement itself. Before we can ask whether capabilities emerge discontinuously, we must ask a more basic question: what exactly are our benchmarks measuring?