Hallucination Mitigation: RAG, Decoding, and Training › Measuring Hallucination Mitigation

Deploying mitigation strategies without measuring their effect is flying blind. Effective evaluation requires benchmarks that test the specific failure modes your mitigation is designed to address, and metrics that capture both the rate of hallucination and the model's calibration.

Several benchmarks have become standard for evaluating factual accuracy in language models:

TruthfulQA tests whether models give truthful answers to questions that many humans answer incorrectly due to common misconceptions. A model that merely reproduces training data will score poorly, because training data contains the misconceptions. The benchmark specifically targets the tendency of capable models to generate plausible-sounding falsehoods.

FELM (Factuality Evaluation of Large Language Models) evaluates factuality across multiple domains (science, medicine, finance, law) by having models generate responses and then checking specific factual claims against verified sources. FELM provides domain-specific breakdowns, making it useful for identifying where a model has strong versus weak factual coverage.

FActScore (described earlier in the Post-Hoc Verification section) evaluates long-form generation by decomposing responses into atomic claims and verifying each one. This is particularly useful for evaluating mitigation strategies on biography, entity description, and summarization tasks where hallucination is fine-grained rather than binary.

HaluEval focuses on hallucination in dialogue and question answering, with tasks that test whether the model can identify hallucinated versus faithful responses. It is useful both for evaluation and for constructing training data for DPO-based mitigation.

Measuring whether a model's expressed uncertainty is calibrated requires comparing stated confidence to empirical accuracy. The standard tool is an expected calibration error (ECE) analysis:

$$\text{ECE} = \sum_{b=1}^{B} \frac{|B_b|}{n} \left| \text{acc}(B_b) - \text{conf}(B_b) \right|$$

where:

• $B_b$: a bin of predictions with confidence scores falling in the same range
• $\text{acc}(B_b)$: the actual accuracy within bin $B_b$
• $\text{conf}(B_b)$: the average confidence within bin $B_b$
• $n$: the total number of predictions

A model with ECE close to 0 is well-calibrated: when it says it is 80% confident, it is right 80% of the time. A model with high ECE is miscalibrated: its stated confidence does not track its actual accuracy. Most language models are overconfident in their factual claims, meaning ECE analysis will show that their high-confidence claims are correct less often than their confidence implies.

For production systems, the most meaningful evaluation is end-to-end: measure the rate of factual errors on a representative sample of real user queries with ground truth labels. This is expensive (it requires human annotation at scale), but it is the only evaluation that captures the full distribution of failure modes in your deployment context.

A practical compromise is periodic automated evaluation on a curated validation set, with human spot-checking to catch systematic errors that the automated metrics miss. The validation set should be refreshed periodically to include new query types and to avoid overfitting mitigation strategies to a fixed benchmark.