Test-Time Compute › Test-Time Compute

Training a language model is expensive. You run billions of gradient updates over trillions of tokens, burn weeks of GPU time, and produce a set of weights that are frozen before the model ever sees a real user query. After that, every inference call uses exactly the same amount of compute, regardless of whether the question is "What is 2 + 2?" or "Prove the Riemann hypothesis." This uniformity is economical but wasteful: trivial questions receive the same resources as intractable ones.

Test-time compute refers to the deliberate allocation of additional computation at inference time to improve output quality. Instead of accepting whatever the model produces in a single forward pass, you spend more FLOPS, generate more candidates, or let the model iterate over its own answers. The key insight is that some problems are easier to verify than to solve, and a model that can check its own work, even imperfectly, can use that signal to self-improve during the generation process.

This chapter examines the main strategies for scaling test-time compute: generating and ranking multiple samples, applying iterative self-refinement, designing compute-optimal inference budgets, and the emerging empirical scaling laws that govern how answer quality changes as you spend more inference FLOPS. As we explored in the preceding chapters on Process Reward Models and Constitutional AI, giving a model the ability to evaluate its own output is a prerequisite for most of these techniques to work.