Evaluation Fundamentals › Evaluation Best Practices

Good evaluation practice requires attention to both methodological rigor and practical constraints. The most important principles are detailed below.

No single metric captures the full picture of model quality. Use a combination of:

• Automatic metrics for fast iteration during development
• Benchmark suites for standardized comparison with other models
• Human evaluation for high-stakes quality assessments
• Task-specific metrics tailored to your deployment use case

Convergence across multiple evaluation approaches increases confidence in conclusions. Divergence signals that different approaches are measuring different things, which is worth understanding. When automatic metrics and human judgments disagree, the disagreement is often informative: it reveals that the automatic metric has a blind spot that matters for the specific task or capability being evaluated.

Point estimates of benchmark performance without confidence intervals are difficult to interpret. Small differences in reported numbers may not be statistically meaningful. Always report confidence intervals, standard deviations across multiple evaluation runs, or results of statistical significance tests when comparing models.

Reporting practices in the research community have historically been poor in this regard. Many benchmark comparisons treat 1-2 point differences as meaningful when they are well within the margin of sampling variability. The broader adoption of statistical reporting would improve the quality of conclusions drawn from evaluation studies, and practitioners reading published results should always ask: how large is the standard error here, and would this difference hold on a different test set?

Evaluation results are only reproducible if all conditions are documented:

• The exact prompt templates used for each task
• Decoding parameters (temperature, top-p, max tokens)
• Model checkpoint and version
• Test set version and any filtering applied
• Scoring code and tie-breaking rules

Undocumented evaluation conditions are a major reason why published results cannot be reproduced and why different teams report different scores for the same model on the same benchmark. When a research paper reports a new state-of-the-art result, the evaluation conditions are often as important as the model architecture for explaining the improvement.

Goodhart's Law states that when a measure becomes a target, it ceases to be a good measure. In LLM evaluation, this manifests as models being fine-tuned specifically on benchmark data (even without explicit contamination) or as evaluation pipelines being designed to produce favorable numbers rather than accurate measurements.

Building evaluation suites that are difficult to game requires continuous refresh of test data, diversity of evaluation formats that cannot all be simultaneously optimized, and honest acknowledgment that benchmark performance is a proxy, not the goal.

A practical corollary is to maintain held-out evaluation sets that the training team does not know about. If the development team never sees the test set, they cannot optimize for it, whether intentionally or inadvertently through choices about training data curation and instruction tuning design.

The most commonly violated principle of evaluation is the simplest: evaluate on the distribution that matters. A coding assistant evaluated on mathematical reasoning benchmarks may show impressive numbers while failing on its actual use case. Construct evaluation sets that represent real queries from real users whenever possible. Usage logs from deployed systems, or user studies with realistic tasks, provide far more relevant signal than academic benchmarks for production systems.

As a practical workflow, teams building production systems should maintain two separate evaluation sets: a public benchmark suite for comparison with the research community and for tracking broad capability trends, and a private deployment evaluation set built from real usage data for making production decisions. The public benchmarks provide context and enable external validation. The private deployment set provides the signal that matters for the product.