Evaluation Challenges › Code Implementation
Language AI HandbookAll chapters Evaluation Challenges › Code ImplementationOne section · Sections in this chapter7 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let's implement a practical evaluation framework that addresses several of the challenges discussed above, including measurement of prompt sensitivity and contamination detection.
Prompt Sensitivity Measurement
We will compare two models: one clean and one with contamination inflating its scores. By evaluating both across multiple prompt formats, we can see how single-format evaluation can produce misleading rankings.
from collections import defaultdict
# Five evaluation questions with known correct answers
questions = [
{
"id": "q1",
"question": "What is the capital of France?",
"correct": "Paris",
},
{
"id": "q2",
"question": "What element has atomic number 8?",
"correct": "Oxygen",
},
{
"id": "q3",
"question": "In what year did World War II end?",
"correct": "1945",
},
{
"id": "q4",
"question": "What is the speed of light in m/s?",
"correct": "3e8",
},
{"id": "q5", "question": "Who wrote Hamlet?", "correct": "Shakespeare"},
]
# Three prompt formats for the same questions
prompt_formats = {
"direct": "Question: {q}\nAnswer:",
"polite": "Please answer the following question: {q}\nYour answer:",
"cot": "Think step by step and answer the following: {q}\nAnswer:",
}
def simulate_model_response(
prompt_format, model_capability, contamination_bias=0.0
):
"""Simulate a model response with capability, contamination, and format effects."""
effective_prob = min(model_capability + contamination_bias, 1.0)
# Different prompt formats produce slightly different accuracies
format_effects = {"direct": 0.0, "polite": -0.05, "cot": 0.08}
effective_prob += format_effects.get(prompt_format, 0.0)
effective_prob = np.clip(effective_prob, 0.0, 1.0)
return np.random.random() < effective_prob
# Two models: clean (true capability 0.70) and contaminated (true 0.65, bias +0.18)
models = {
"Model_A_clean": {"capability": 0.70, "contamination": 0.0},
"Model_B_contaminated": {"capability": 0.65, "contamination": 0.18},
}
n_trials = 200 # Repeat each question many times for stable estimates
results = defaultdict(lambda: defaultdict(list))
for model_name, model_params in models.items():
for fmt_name in prompt_formats:
for _ in range(n_trials):
for q in questions:
correct = simulate_model_response(
fmt_name,
model_params["capability"],
model_params["contamination"],
)
results[model_name][fmt_name].append(correct)
# Summarize accuracy and standard error per model per format
accuracy_table = {}
for model_name in models:
accuracy_table[model_name] = {}
for fmt_name in prompt_formats:
scores = results[model_name][fmt_name]
accuracy_table[model_name][fmt_name] = {
"mean": np.mean(scores),
"std": np.std(scores) / np.sqrt(len(scores)),
}Evaluation Results by Model and Prompt Format ============================================================== Format Model A (Clean) Model B (Contam.) -------------------------------------------------------------- direct 0.711 ± 0.014 0.846 ± 0.011 polite 0.632 ± 0.015 0.787 ± 0.013 cot 0.771 ± 0.013 0.897 ± 0.010 Model_A_clean: avg=0.705, prompt sensitivity (range)=0.139 Model_B_contaminated: avg=0.843, prompt sensitivity (range)=0.110
The output shows that Model B's contamination gives it a higher score on the direct prompt, making it look superior. But when you average across prompt formats, Model A's higher true capability gives it a better mean score. Model B also shows greater prompt sensitivity (a larger gap between its best and worst prompt format scores), which is consistent with its performance being partly driven by matching memorized patterns that work better with one format than another.
This simulation compresses into a few lines a dynamic that plays out across the whole field. A researcher who evaluates Model B only with the direct prompt format will conclude it is the better model. A more careful evaluator who tests multiple formats will reach the opposite conclusion. The gap between these two conclusions is not about which evaluator worked harder; it is about whether evaluation was designed as a scientific measurement or as a score retrieval exercise.
Contamination Detection via N-gram Overlap
A common contamination detection strategy is measuring how many n-grams from a test item appear in the training corpus. Items with high overlap are flagged as potentially contaminated. This approach scales to large corpora and can be applied without running the model, making it a useful preprocessing step before any evaluation.
def get_ngrams(text, n):
"""Extract n-grams from a text string."""
words = text.lower().split()
return [tuple(words[i : i + n]) for i in range(len(words) - n + 1)]
def contamination_score(test_item, training_corpus, n=3):
"""
Compute the fraction of n-grams in test_item that appear in training_corpus.
Higher scores indicate potential contamination.
"""
test_ngrams = set(get_ngrams(test_item, n))
if not test_ngrams:
return 0.0
training_ngrams = set()
for doc in training_corpus:
training_ngrams.update(get_ngrams(doc, n))
overlap = test_ngrams & training_ngrams
return len(overlap) / len(test_ngrams)
# Simulated training corpus: documents covering various factual topics
training_docs = [
"The capital of France is Paris a city on the Seine river",
"Paris is the capital and most populous city of France",
"Oxygen has atomic number 8 and is essential for combustion",
"Shakespeare was an English playwright who wrote many famous works",
"The speed of light in vacuum is approximately three times ten to the eighth meters per second",
"World War II ended in nineteen forty five with the surrender of Germany and Japan",
"Some unrelated document about cooking recipes and kitchen utensils",
"Another document about programming languages and software development practices",
]
# Test items formed from our question set
test_items = [
"What is the capital of France Paris is the correct answer",
"What element has atomic number 8 it is oxygen",
"In what year did World War II end 1945 was the final year",
"What programming language was invented in the early 1990s",
"Who wrote the play Hamlet it was William Shakespeare",
]
contam_threshold = 0.30
contam_scores = [
(item[:42] + "...", contamination_score(item, training_docs, n=3))
for item in test_items
]Contamination Detection Results (3-gram overlap with training corpus) ==================================================================== Test Item (truncated) Score Status -------------------------------------------------------------------- What is the capital of France Paris is the... 0.444 FLAGGED What element has atomic number 8 it is oxy... 0.286 clean In what year did World War II end 1945 was... 0.091 clean What programming language was invented in ... 0.000 clean Who wrote the play Hamlet it was William S... 0.000 clean Flagged: 1/5 items (20%) at threshold 30%
Items about well-documented facts (Paris as capital, World War II) are flagged because the training corpus contains text about those topics. The programming language question scores low because nothing similar appears in training. In production contamination detection, this logic is applied at much larger scale, often using suffix lengths of 50 or more tokens and corpora of billions of documents.
The key insight from this detection exercise is that flagging does not tell you whether a model's correct answers on flagged items came from memorization or from learned knowledge. A model that knows the capital of France because it was trained on a document mentioning Paris is not in the same situation as a model that memorized a specific benchmark question and its answer. But from the perspective of evaluation, both situations create the same problem: the model may have an unfair advantage on those items that inflates its measured accuracy relative to its performance on truly novel questions.
Now let's visualize how prompt brittleness affects model rankings when evaluated across multiple prompt variants:
n_models = 5
n_prompt_variants = 10
true_caps = np.array([0.74, 0.70, 0.71, 0.68, 0.66])
# Brittleness: std of performance across prompt variants
brittleness = np.array([0.04, 0.14, 0.08, 0.16, 0.06])
model_names_short = [f"M{i + 1}" for i in range(n_models)]
# Structured prompt effects preserve each model's mean while making brittle models
# more likely to look unusually strong under one favorable formulation.
prompt_effects = np.array(
[
[-1.2, -0.8, -0.5, -0.3, -0.1, 0.1, 0.3, 0.5, 0.8, 1.2],
[-1.6, -1.2, -0.8, -0.5, -0.2, 0.1, 0.3, 0.6, 1.2, 2.1],
[-1.5, -1.0, -0.6, -0.3, -0.1, 0.1, 0.3, 0.5, 0.9, 1.7],
[-1.8, -1.2, -0.8, -0.4, -0.1, 0.1, 0.4, 0.8, 1.2, 1.8],
[-1.3, -0.9, -0.5, -0.2, -0.1, 0.1, 0.2, 0.5, 0.9, 1.3],
]
)
scores_by_prompt = np.clip(
true_caps[:, None] + brittleness[:, None] * prompt_effects, 0, 1
)
mean_scores = scores_by_prompt.mean(axis=1)
# Rankings: by best single-prompt score vs. mean across all prompts
rank_by_best = np.argsort(-scores_by_prompt.max(axis=1))
rank_by_mean = np.argsort(-mean_scores)
The box plots make prompt brittleness tangible. Models M2 and M4 have wide boxes and long whiskers, indicating high variance across prompt variants. Under a single favorable prompt, M2 ranks first. But its mean rank is third. A researcher who evaluated M2 on one lucky prompt would overestimate its capability. A more careful evaluator using 10 prompt variants would correctly identify M1 as the most reliable model.
The annotation showing both best-prompt rank and mean rank captures what multi-prompt evaluation adds beyond single-prompt evaluation. Selecting a model for deployment based on its best single-prompt score would lead you to choose a model that performs inconsistently in the field. Selecting based on mean rank across prompts would lead you to a more reliable choice, even though the peak score is lower.
Computing Confidence Intervals for Benchmark Scores
Let's implement the statistical uncertainty calculation described earlier in the best practices section, making it concrete with both the analytical formula and a bootstrap approach.
from scipy import stats
def analytical_ci(correct, n, confidence=0.95):
"""
Compute a confidence interval for accuracy using the Wilson score method.
Wilson score is more accurate than the normal approximation for extreme probabilities.
"""
p_hat = correct / n
z = stats.norm.ppf((1 + confidence) / 2)
denominator = 1 + z**2 / n
center = (p_hat + z**2 / (2 * n)) / denominator
margin = (
z * np.sqrt(p_hat * (1 - p_hat) / n + z**2 / (4 * n**2))
) / denominator
return center - margin, center + margin
def bootstrap_ci(correct_vector, n_boot=2000, confidence=0.95):
"""
Compute a bootstrap confidence interval for accuracy.
Resamples the binary correct/incorrect vector with replacement.
"""
boot_means = [
np.mean(
np.random.choice(
correct_vector, size=len(correct_vector), replace=True
)
)
for _ in range(n_boot)
]
alpha = (1 - confidence) / 2
return np.percentile(boot_means, [alpha * 100, (1 - alpha) * 100])
# Simulate benchmark results for two models on a 500-item benchmark
n_items = 500
model_x_correct = np.random.binomial(1, 0.72, n_items) # true p=0.72
model_y_correct = np.random.binomial(1, 0.75, n_items) # true p=0.75
model_x_acc = model_x_correct.mean()
model_y_acc = model_y_correct.mean()
# Confidence intervals using both methods
x_analytical = analytical_ci(model_x_correct.sum(), n_items)
y_analytical = analytical_ci(model_y_correct.sum(), n_items)
x_bootstrap = bootstrap_ci(model_x_correct)
y_bootstrap = bootstrap_ci(model_y_correct)Model X: accuracy = 0.730 Analytical 95% CI: (0.689, 0.767) Bootstrap 95% CI: (0.692, 0.766) Model Y: accuracy = 0.732 Analytical 95% CI: (0.692, 0.769) Bootstrap 95% CI: (0.692, 0.772) CI overlap: 0.076 The confidence intervals overlap -- the difference may not be statistically significant.
The confidence interval chart makes explicit what a single reported number hides. Model Y's accuracy appears higher than Model X's, but if the intervals overlap, that difference is consistent with sampling variation alone. Reporting only point estimates without error bars creates an illusion of precision that the data do not support and leads readers to draw conclusions that are not warranted.