Kaplan Scaling Laws › Key Parameters
Michael BrenndoerferJanuary 13, 20251 min read
Language AI HandbookAll chapters Part 23 of 67Scaling Laws
Kaplan Scaling Laws › Key ParametersOne section · Sections in this chapter14 / 15 ↓
Sections in this chapter15
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for Kaplan scaling laws are:
- α_N (0.076): Parameter scaling exponent. Determines how quickly loss decreases as model size increases.
- α_D (0.095): Data scaling exponent. Governs the rate of loss improvement with more training tokens.
- α_C (0.050): Compute scaling exponent. Controls loss reduction when compute budget increases with optimal allocation.
- N_c, D_c, C_c: Characteristic scale constants. Fitted values representing the scale at which each resource reaches a reference loss level.
- Allocation exponents (0.73, 0.27): Optimal compute allocation. Specify how to split compute budget between larger models and more data.