Evaluation Challenges › Contamination and Brittleness in Practice
Language AI HandbookAll chapters Evaluation Challenges › Contamination and Brittleness in PracticeOne section · Sections in this chapter5 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
To make the impact of contamination and prompt sensitivity concrete, let's build a small simulation. We will compare two models: one clean and one with contamination inflating its scores. We will also visualize how reported benchmark scores can diverge from true capability, and how benchmark saturation plays out across successive generations of benchmarks.
import numpy as np
# Six hypothetical models with different true capabilities
true_capabilities = np.array([0.55, 0.62, 0.68, 0.72, 0.78, 0.83])
model_labels = [
"Model A",
"Model B",
"Model C",
"Model D",
"Model E",
"Model F",
]
# Contamination inflates reported scores by different amounts
contamination_inflation = np.array([0.0, 0.05, 0.12, 0.18, 0.08, 0.22])
# Prompt brittleness: standard deviation of performance across prompt variants
prompt_brittleness = np.array([0.03, 0.08, 0.05, 0.12, 0.06, 0.10])
# Reported scores = true capability + contamination inflation + small noise
reported_scores = (
true_capabilities
+ contamination_inflation
+ np.random.normal(0, 0.01, len(true_capabilities))
)
# Score range across prompt variants (mean +/- 2 std)
low_scores = reported_scores - 2 * prompt_brittleness
high_scores = reported_scores + 2 * prompt_brittleness
Models D and F appear significantly more capable than their underlying capability warrants because contamination inflation boosts their reported scores. At the same time, the wide error bars on models B and D reveal high prompt brittleness: their apparent performance could be quite different depending on which prompt format an evaluator happens to use. Two evaluators using different prompt formats for Model D might reach opposite conclusions about whether it outperforms Model C.
Now let's simulate benchmark saturation: the pattern where model performance rises rapidly to match human baselines, forcing the creation of harder replacement benchmarks.
years = np.linspace(2018, 2026, 100)
def sigmoid_score(years, center, steepness, asymptote=0.98):
"""Logistic growth curve modeling rapid benchmark saturation."""
return asymptote / (1 + np.exp(-steepness * (years - center)))
# Three successive benchmark generations
glue_scores = sigmoid_score(years, center=2019.5, steepness=3.5, asymptote=0.97)
superglue_scores = sigmoid_score(
years, center=2021.0, steepness=3.2, asymptote=0.96
)
bigbench_scores = sigmoid_score(
years, center=2022.8, steepness=3.8, asymptote=0.94
)
# Approximate human performance baselines for each benchmark
glue_human = 0.87
superglue_human = 0.89
bigbench_human = 0.80
The saturation curves illustrate the treadmill dynamic. Each benchmark provides a useful window of differentiation before model performance catches up. The dotted lines show approximate human baselines, and models surpass human-level performance on each benchmark within a few years. This pattern has driven the creation of progressively harder evaluation suites and has accelerated the field's search for benchmarks that remain informative.
Notice, too, that the human baselines differ across benchmarks. BIG-Bench Hard was specifically designed to contain tasks that are difficult for humans as well as models, which is part of why its human baseline sits lower than GLUE or SuperGLUE. This design choice slows saturation but also raises questions about what "superhuman" performance means on a benchmark where humans score 80%: is the model better, or has it found patterns in the benchmark structure that humans would not exploit given enough time?