Hallucination Detection: NLI, Self-Consistency & Learned Models › Evaluation Benchmarks

Before deploying any hallucination detector, you need to know how well it performs on data that resembles your actual use case. A rich ecosystem of benchmarks has emerged for this purpose, each targeting different aspects of the detection problem.

AggreFact is a benchmark for summarization factuality that aggregates annotations from multiple prior datasets, covering both abstractive summarizers and more recent language models. It distinguishes between different types of factual errors, such as entity errors and relational errors, and provides a large pool of human judgments for calibrating detector thresholds. Models that perform well on AggreFact tend to transfer reasonably to real production settings for news summarization tasks.

TRUE Benchmark evaluates detectors across multiple tasks simultaneously, covering summarization, dialogue, QA, and fact verification. Because it tests the same model on all tasks, it penalizes models that specialize in one domain at the expense of others. This makes it particularly useful for evaluating general-purpose detectors like TRUE or MiniCheck.

FELM (Fine-grained Evaluation of Language Model Factuality) focuses on a broader range of factual categories, including world knowledge, reasoning, and instructions, rather than just document-grounded claims. It exposes failure modes that document-only benchmarks miss, particularly on claims that require multi-hop reasoning or combining facts from multiple sources.

FActScore Benchmark is the gold standard for open-domain biographical factuality. By collecting detailed human annotations on LLM-generated biographies of people at different levels of fame, it reveals how model accuracy degrades as subjects become less prominent in training data. Models are much more accurate about famous historical figures than about minor figures who appeared infrequently in pretraining corpora.

When evaluating a detector, report area under the ROC curve (AUC) in addition to accuracy at a fixed threshold, since the optimal threshold will vary across applications. Also report false positive and false negative rates separately, because the cost of each error type differs: a false positive wastes a user's trust by flagging a correct response, while a false negative damages trust by letting a hallucination through.