Evaluation Fundamentals › Evaluation Metrics Overview

Evaluation metrics convert model outputs into numerical scores. The choice of metric is as important as the choice of test set, because the metric determines what "better" means in your evaluation. There are several broad families of metrics, each with distinct strengths and weaknesses.

Reference-based metrics compare model outputs against gold-standard reference answers. They are fast, cheap, and reproducible, making them standard for large-scale evaluation during development.

The simplest reference-based metrics are exact match and token-level overlap. Exact match is appropriate when there is a single correct answer (factual questions, code that must have a specific output), and it gives full credit for correct answers and zero credit for everything else. Token overlap metrics like BLEU and ROUGE measure partial credit by comparing n-gram sequences between model output and reference. As we will examine in detail in the Automatic Metrics chapter, these metrics correlate reasonably with human judgments for tasks like machine translation and summarization, but they have well-known failure modes: they reward surface-level similarity over semantic correctness and penalize valid paraphrases.

Semantic similarity metrics like BERTScore address the surface-level limitation by comparing representations in embedding space rather than token sequences. A model output that is semantically equivalent but lexically different from the reference will receive a higher BERTScore than it would from BLEU or ROUGE.

The brevity penalty in BLEU deserves particular attention. Without it, you could achieve 100% precision trivially by generating a single word that appears in the reference. The brevity penalty closes this loophole by requiring that generated text have approximately the same length as the reference. But this introduces a new problem: it assumes the reference is the right length, which is not always true. A model that generates a concise but complete summary will be penalized relative to one that generates a verbose summary that pads out to reference length, even if the concise summary is better.

These limitations are fundamental rather than accidental. N-gram overlap metrics were designed for specific tasks with well-defined references, and they work reasonably well there. When applied outside those settings, their limitations become dominant. The research community has developed a variety of alternatives, from BERTScore and BLEURT (which use contextual embeddings) to task-specific metrics (which measure what matters for the task), but automatic metrics remain an approximation. They are valuable for the speed and scale they enable, not as ground truth about quality.

Many evaluation tasks have established, purpose-built metrics. Some important examples:

Accuracy and F1 for classification tasks. Accuracy measures the fraction of examples classified correctly. F1 balances precision and recall, which matters when class distributions are imbalanced or when false positives and false negatives have different costs.

Exact match and character error rate for extraction and transcription tasks. These measure whether the model extracts precisely the right span of text from a document, or transcribes speech with minimal character-level error.

Execution success rate and test pass rate for code generation. A code model is most naturally evaluated by whether the code it generates runs and produces correct outputs, not by how similar the generated code looks to a reference solution.

Factual accuracy and citation correctness for retrieval-augmented generation. These require external verification against a knowledge base or documents, which adds infrastructure cost but provides more meaningful signals than surface-level comparisons.

Task-specific metrics are more meaningful than general automatic metrics precisely because they measure the right thing for the specific task. Code execution rate is a far better measure of coding ability than BLEU similarity to a reference solution, because two code snippets can be functionally identical while being textually very different, and BLEU would penalize this. Likewise, for structured output tasks like slot filling, exact match on the extracted fields is far more informative than token overlap.

The cost of task-specific metrics is that they require custom infrastructure. You need a code execution environment with the right language versions, libraries, and sandboxing. You need a knowledge base and fact-checking pipeline. You need structured output parsers that can handle format variations. This infrastructure takes time to build and maintain, which is why teams often start with general automatic metrics and graduate to task-specific metrics as they scale.

Human evaluation asks people to judge model outputs directly, using scales like Likert ratings, preference comparisons, or detailed rubrics. It is the most direct way to measure subjective qualities like helpfulness, fluency, coherence, and appropriateness, which automatic metrics struggle to capture.

Human evaluation is expensive and time-consuming. A single evaluation study involving hundreds of examples and multiple annotators can cost thousands of dollars and take weeks. This limits how frequently it can be used during development. It is best reserved for major milestones: comparing production candidates, measuring alignment properties, or validating that automatic metrics are tracking human judgments correctly.

Inter-annotator agreement, measured using Cohen's kappa or Krippendorff's alpha, is a critical diagnostic for human evaluation. Low agreement means annotators are not measuring the same thing, which makes the results unreliable. Agreement should always be reported alongside human evaluation results.

Human evaluation is also subject to its own biases that are easy to overlook. Annotators are influenced by the order in which responses are presented. They apply different standards depending on the context and their prior experience. They may agree with well-written prose even when it is factually wrong, because fluency and accuracy are not the same thing. Designing annotation protocols that are clear, detailed, and tested with pilot studies before the main evaluation is essential for getting reliable results.

The most common failure mode in human evaluation is giving annotators a task that is underspecified. "Rate this response on a scale of 1 to 5 for helpfulness" seems clear, but different annotators will apply different implicit criteria. One annotator might focus on whether the response is factually accurate. Another might focus on whether it directly addresses the question. A third might weight tone and politeness. Without a detailed rubric that specifies what each score level means and what dimensions to consider, annotators will diverge, agreement will be low, and the evaluation will produce noisy results. Invest time in rubric design before running any human evaluation.

A newer evaluation paradigm uses strong language models to rate the outputs of weaker or different models. GPT-4 evaluating GPT-3.5 outputs, for example, or using a dedicated judge model to score responses on a rubric. This approach is faster and cheaper than human evaluation while correlating reasonably with human judgments on many dimensions.

The appeal of LLM-as-judge is that language models can assess properties like helpfulness, coherence, and instruction-following that are difficult to measure with n-gram overlap but easy for a capable model to recognize. A model can understand that two responses convey the same information even when they use different words, and it can assess whether a response follows the spirit of a request rather than just its literal wording. These are exactly the qualities where automatic metrics fail and human evaluation is expensive.

LLM-as-judge has notable failure modes. Strong position bias causes judge models to systematically prefer the first answer when two options are presented, even when the two answers are randomly swapped. Verbosity bias causes judge models to prefer longer answers regardless of quality, rewarding quantity over correctness. Self-enhancement bias causes a judge model to prefer outputs that resemble its own generation style, creating systematic favoritism when the judge and one of the evaluated models share architecture or training data.

These biases must be mitigated through careful evaluation design. Presenting each pair in both orders and averaging the scores addresses position bias. Explicitly instructing the judge to value conciseness and checking that brevity is not penalized addresses verbosity bias. Using human-written reference answers or multiple diverse judge models addresses self-enhancement bias. We will examine this evaluation paradigm in depth in the LLM-as-Judge chapter, including experimental evidence on how large these biases are and which mitigation strategies work best.

Despite its limitations, LLM-as-judge has become a practical workhorse for model evaluation, particularly for instruction-following and open-ended generation tasks where no good automatic metric exists. Systems like MT-Bench and AlpacaEval use LLM-as-judge as their primary evaluation mechanism, and this has enabled rapid iteration on model quality during post-training at a fraction of the cost of human evaluation. The key is to treat LLM-as-judge scores as estimates with known biases that require calibration, not as ground truth.

Benchmark suites like MMLU, HellaSwag, and ARC bundle hundreds or thousands of multiple-choice questions across diverse domains into standardized evaluation protocols. They enable rapid comparison across many models and have become the standard way to track progress in the research community.

The strength of benchmark suites is their breadth and standardization. The weakness is that they can become outdated as models improve, and performance improvements on a benchmark may not reflect real capability improvements if the benchmark has been contaminated or if the model has been specifically tuned to perform well on it. We will examine major benchmark suites and their design properties in the Benchmark Design chapter.

The figure below positions the main evaluation approaches across two key dimensions: evaluation cost and alignment with human judgment. No single approach dominates; the choice depends on which tradeoff matters most for your current development stage.