Emergence vs Metrics › Key Parameters
Michael BrenndoerferNovember 8, 20251 min read
Language AI HandbookAll chapters Part 30 of 67Emergent Capabilities
Emergence vs Metrics › Key ParametersOne section · Sections in this chapter12 / 13 ↓
Sections in this chapter13
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters and concepts for understanding emergence metrics are:
- per_token_prob (p): The probability of generating each correct token. Small improvements in per-token probability can produce dramatic changes in sequence accuracy.
- sequence_length (n): The number of tokens in the target output. Longer sequences amplify threshold effects exponentially.
- Brier score: A continuous metric measuring mean squared error between predictions and outcomes. Unlike accuracy, it captures improvement even below decision thresholds.
- threshold: The decision boundary (typically 0.5) where predictions flip from negative to positive class. Accuracy only changes when probabilities cross this boundary.
The next chapter examines inverse scaling, a counterintuitive phenomenon where larger models sometimes perform worse on specific tasks. This challenges even the basic assumption that more scale produces more capability.