Hallucination Detection: NLI, Self-Consistency & Learned Models › Entailment-Based Detection

The most principled approach to hallucination detection frames the problem as a Natural Language Inference (NLI) task. NLI is the problem of deciding, given a premise $p$ and a hypothesis $h$, whether $p$ entails $h$, contradicts $h$, or is neutral with respect to $h$. This framing captures exactly what we care about in hallucination detection: we want to know whether the model's claims are supported by, contradicted by, or simply absent from some reference text.

Applied to hallucination detection, the premise is the source document or retrieved context, and the hypothesis is a claim from the model's generated output. If the source entails the claim, the claim is likely grounded. If the source contradicts the claim, the claim is almost certainly a hallucination. Neutral relationships are more ambiguous: the claim may be true but unsupported by the available source, or it may be false but simply not addressed by the source. For high-stakes applications, both contradiction and neutral often warrant further inspection.

Why use NLI rather than simpler similarity measures like cosine similarity or ROUGE? Because similarity is not the same as factual support. A generated summary could be semantically similar to a source document while still containing false claims, simply because it uses the same vocabulary and covers the same topic. NLI models are explicitly trained to reason about logical relationships, not surface overlap. They can recognize that "Curie won the Nobel Prize in 1905" contradicts "She won the Nobel Prize in 1903" even though both sentences are about the same topic and share most of their words. The NLI framing captures this directional, logical relationship that similarity metrics miss.

An NLI model assigns probabilities over three mutually exclusive labels. Given a source document $D$ and a generated claim $c$, the model $f$ produces:

where:

• $D$: the source document or reference passage used as the premise
• $c$: the generated claim being evaluated as the hypothesis
• $p_{\text{entail}}$: probability that the source logically implies the claim
• $p_{\text{neutral}}$: probability that the source neither implies nor contradicts the claim
• $p_{\text{contradict}}$: probability that the source implies the negation of the claim
• $p_{\text{entail}} + p_{\text{neutral}} + p_{\text{contradict}} = 1$ by construction

The factual consistency score for claim $c$ is typically taken as the entailment probability, since this directly measures how well the source supports the claim:

where a score near 1 means the source strongly supports the claim, and a score near 0 means the claim is unsupported or contradicted. In cases where you want to specifically flag contradictions rather than just low entailment, you can threshold on $p_{\text{contradict}}$ directly, which behaves differently from $1 - p_{\text{entail}}$ because neutral claims will not be caught by the contradiction signal.

For a full generated summary or response $G$ decomposed into $n$ sentences $\{c_1, c_2, \ldots, c_n\}$, you compute a document-level score by aggregating the per-sentence scores. Two common aggregation strategies are the minimum and the mean:

where:

• $n$: number of sentences in the generated output
• $c_i$: the $i$-th sentence of the generated output
• $p_{\text{entail}}(D, c_i)$: entailment probability for that sentence against the source

The minimum-based score is more conservative: a single unsupported sentence tanks the overall score, making it well-suited for high-stakes applications where any fabrication is unacceptable. The mean is more forgiving and often correlates better with human judgment on longer documents where minor imprecisions are expected. A practical approach is to report both and use the minimum as an alert threshold while using the mean as a quality signal.

There is also a third option, weighted aggregation, where sentences containing named entities, numbers, or dates receive higher weight because they carry the highest hallucination risk. This requires a lightweight tagger to identify high-stakes claims before scoring but can improve correlation with human judgments on documents where factual precision matters unevenly across sentences.

SummaC (Summarization Factual Consistency) formalizes the entailment idea into a practical detection framework that handles long source documents more gracefully. Rather than scoring each generated sentence against the full source concatenated together, SummaC scores each generated sentence against each source sentence individually and builds a score matrix.

The key motivation for this design is that source documents are often long and structured. A question-and-answer document might have dozens of paragraphs, but a particular generated claim may only be supported by one specific paragraph. If you concatenate the entire source and pass it to an NLI model, you run into the model's context limit and dilute the signal: the entailment score for a specific claim gets averaged across many irrelevant source sentences. SummaC solves this by finding the single source sentence that best supports each generated sentence, which is both more accurate and more interpretable. It also sidesteps the context length problem entirely, since each NLI call processes only a single pair of sentences.

Let the source be split into $m$ sentences $\{s_1, \ldots, s_m\}$ and the generated output into $n$ sentences $\{c_1, \ldots, c_n\}$. The score matrix $M \in \mathbb{R}^{n \times m}$ captures all pairwise entailment scores:

where:

• $M_{ij}$: entailment probability that source sentence $s_j$ supports generated sentence $c_i$
• $s_j$: the $j$-th sentence of the source document, used as the premise
• $c_i$: the $i$-th sentence of the generated output, used as the hypothesis
• $n \times m$: the matrix has one row per generated sentence and one column per source sentence

For each generated sentence $c_i$, the row score is the maximum entailment score across all source sentences:

where:

• $\text{row}_i$: how well the best-matching source sentence supports generated sentence $c_i$
• The maximum is taken because a claim only needs to be supported by one part of the source to be grounded. Averaging would penalize claims that are tightly supported by a specific section simply because most other source sentences are irrelevant to that claim.

The final document-level score is the mean of row scores:

where:

• $\text{SummaC}(G) \in [0, 1]$: the overall factual consistency score for the generated text
• Values near 1 indicate all generated sentences are well-supported by the source
• Values near 0 indicate many generated sentences lack source support

The construction of the score matrix also gives you interpretability for free. When a sentence receives a low row score, you can inspect the row to see which source sentences were checked and confirm that none of them supports the claim. This makes the detection system auditable, which is valuable in applications where you need to explain why a response was flagged.

SummaC significantly outperformed prior metrics like ROUGE on factual consistency benchmarks, particularly on cases where the summary is fluent and semantically similar to the source but contains subtle factual errors. The improvement is most pronounced on long source documents and on numerical claims, where surface similarity is a poor proxy for factual accuracy.

The quality of entailment-based detection depends heavily on which NLI model you use as the backbone. This choice affects accuracy, generalization across domains, sensitivity to numerical claims, and handling of long inputs. Three models are worth understanding in depth because they represent different points on the generalization-precision tradeoff.

DeBERTa-v3-large-mnli is the strongest general-purpose option. DeBERTa improves on BERT by using disentangled attention, which encodes content and positional information separately and allows the model to better capture fine-grained semantic relationships, including the kind of subtle logical dependencies that distinguish entailment from neutral. Trained on the MultiNLI dataset, which covers fiction, telephone transcripts, travel guides, government documents, and other genres, this model generalizes well but was not specifically optimized for factuality in machine-generated text. It is the right choice for new domains where no fine-tuned model exists and you are willing to accept slightly lower precision in exchange for broad coverage.

FactCC trades generalization for precision on document-summarization factuality. By training on synthetically perturbed news articles, FactCC learns the specific kinds of errors that summarization models make, including entity swaps, pronoun confusions, and subtle numerical changes. On standard summarization factuality benchmarks covering XSum and CNN/DailyMail, FactCC significantly outperforms generic NLI models. However, because it was trained entirely on news data, it can underperform on domains with very different writing styles, such as scientific papers or dialogue. If your application involves news summarization specifically, FactCC is likely the best choice; if not, it may be over-specialized.

TRUE is the best choice when you need to generalize across diverse input types. By training on a mixture of NLI, dialogue, question-answering, and fact-checking datasets, TRUE builds a broader understanding of what "factual support" means in different contexts. If your system handles both document summarization and conversational grounding, TRUE generalizes better than a single-domain model. Its multi-task training effectively regularizes the model against overfitting to any single dataset's distributional quirks.

In practice, the right approach is to evaluate two or three backbone options on a small labeled sample from your specific domain and use the model that best matches human judgments on that data. Swapping out the backbone is straightforward because the interface is always the same: a (premise, hypothesis) pair in, a three-way probability distribution out. The modular design of entailment-based detection means you can improve accuracy by simply upgrading the backbone without changing any other part of the pipeline.