Test-Time Compute › Summary

Test-time compute is one of the most impactful levers for improving language model quality without retraining. The key ideas from this chapter are:

• Multiple sampling generates $N$ independent candidates and selects the best via majority vote, reward model scoring, PRM scoring, or direct verification. The pass@k formula characterizes how coverage improves with $N$, following the expression $1 - (1-p)^k$ for independent samples with per-sample accuracy $p$.
• Iterative refinement generates a candidate, evaluates it (via self-critique or external feedback), and revises. It works best when verification is reliable, answers are revisable, and the feedback signal is more accurate than the model's self-assessment. External feedback sources such as code execution and formal verification outperform pure self-critique because they break the correlation between the generator's errors and the critic's blind spots.
• Compute-optimal inference asks how to allocate a fixed token budget between breadth (more samples) and depth (longer reasoning chains). Easy problems prefer greedy decoding; hard verifiable problems prefer PRM-guided reranking with many samples; tasks where reasoning chains substantially improve per-sample accuracy prefer deeper chains over more samples.
• Scaling laws at inference time show sublinear but consistent returns from additional compute, following approximately $\text{Accuracy}(N) \approx a \cdot N^b + c$ with $b \in (0, 1)$. A small model with a large inference budget can match a large model with standard decoding on structured reasoning tasks, suggesting partial substitutability between model capacity and inference compute.
• Adaptive allocation routes queries to different compute budgets based on estimated difficulty, enabling production systems to achieve near-optimal quality at a fraction of the uniform-sampling cost.
• Practical limits include latency, cost, selector quality, diminishing returns, the verification bottleneck on open-ended tasks, and the ceiling imposed by systematic model errors that no amount of resampling can overcome.

As you move forward, the concepts from this chapter connect directly to the Retrieval-Augmented Generation chapter, where external knowledge is a form of test-time compute: instead of sampling more from the model, you query an external corpus and condition generation on the retrieved evidence. This is, in a sense, the most reliable form of external feedback possible, replacing model self-assessment with factual grounding from authoritative sources.