Evaluation Fundamentals › Summary

Evaluation is the systematic measurement of language model capabilities and behaviors. In this chapter, we covered the foundational concepts you need to design and interpret evaluations correctly.

The core evaluation concepts:

• Evaluation goals: tracking progress, selecting models, communicating capabilities, and validating that training objectives generalize to real-world use
• Capability vs. tendency: a model may be capable of something without reliably tending to do it; evaluation design determines which you are measuring
• Intrinsic vs. extrinsic metrics: intrinsic metrics measure model properties in isolation; extrinsic metrics measure task performance that represents real use cases
• Static vs. interactive evaluation: static evaluations present fixed prompts; interactive evaluations measure multi-turn dynamics that matter in agentic deployment
• Behavioral vs. capability evaluation: behavioral evaluation measures what a model does under realistic conditions; capability evaluation measures what it can do under ideal conditions

The key design principles:

• Test set construction requires distribution match, contamination avoidance, diversity, label quality, and temporal and demographic coverage
• Sample size and statistical power must be planned before running evaluations; small samples cannot reliably detect meaningful differences
• Controls for confounds, including prompt sensitivity, decoding parameters, position effects, and model scale, are essential for fair comparisons

The main metric families:

• Automatic reference-based metrics (BLEU, ROUGE, BERTScore) are fast and reproducible but are proxies for quality
• Task-specific metrics (accuracy, F1, execution success rate) measure what directly matters for specific tasks
• Human evaluation is the gold standard for subjective qualities but is expensive and requires inter-annotator agreement measurement
• LLM-as-judge offers a scalable middle ground between automatic metrics and human evaluation, with known biases that must be managed
• Benchmark suites enable standardized comparison but are subject to saturation and contamination

Common failure modes to watch for:

• Benchmark contamination: test data appearing in training data inflates scores without reflecting underlying capability
• Benchmark saturation: ceilings on established benchmarks mask real capability differences between frontier models
• Overfitting to evaluation: training choices that target benchmark performance without improving underlying capability
• Metric-goal misalignment: the metric measures something related to but not identical to what the system is supposed to do

The next chapters in this part build on these foundations, examining each evaluation paradigm in detail. In Automatic Metrics, we will derive the BLEU, ROUGE, and BERTScore formulas from first principles and understand their failure modes precisely. In Human Evaluation, we will cover annotation study design, inter-annotator agreement, and how to get reliable signal from human raters. In LLM-as-Judge, we will examine the best practices for using strong models as evaluators and how to mitigate systematic biases.