Hallucination Detection: NLI, Self-Consistency & Learned Models › Self-Consistency Checks

The approaches above all require some external reference, whether a source document, a knowledge base, or retrieved passages. But what if you do not have access to any of these? Self-consistency methods exploit a property of language models themselves: if a model is uncertain about a fact, its outputs will vary across different samples. If it confidently knows a fact, it should produce consistent outputs regardless of how the question is asked.

The intuition comes from thinking about the model's internal state. A well-trained language model encodes factual information in its parameters as probability distributions over tokens. For facts the model has seen many times in training, the distribution will be sharply peaked, and multiple samples will agree. For facts the model has encountered infrequently or inconsistently, the distribution will be flatter, and samples will diverge. Hallucinations often arise from the model generating plausible-sounding continuations rather than retrieving specific memorized facts. These plausible-sounding outputs tend to be more varied because there are many plausible-sounding falsehoods, but typically only one true answer.

This connection to Bayesian uncertainty is more than just intuition. Self-consistency scores have been shown to correlate with model calibration in several studies. Models that are well-calibrated (meaning their expressed confidence matches their accuracy) also tend to have higher self-consistency on facts they know and lower self-consistency on facts they do not. This suggests that consistency checking is tapping into epistemic uncertainty rather than just measuring output diversity.

SelfCheckGPT formalizes this intuition into a practical detection algorithm. Given a generated passage $G$, the method works as follows:

1. Sample $n$ additional independent passages $\{G_1, G_2, \ldots, G_n\}$ from the same model with the same prompt, using non-zero temperature to ensure diversity
2. For each sentence $c_i$ in $G$, measure how consistently $c_i$'s content appears across the sampled passages

The key point is that sampling is done independently: each $G_j$ is generated from scratch, not conditioned on $G$. If the model repeatedly generates the same fact across independent samples, that fact is likely something the model has encoded reliably. If the model contradicts itself across samples, the claim is less trustworthy. Independence is essential because conditioning on $G$ would introduce strong coherence pressure that would cause the model to repeat the same claims, masking the model's uncertainty.

The consistency score for sentence $c_i$ can be computed in three ways, each with different tradeoffs between cost and accuracy.

NLI-based consistency measures how often the sampled passages contradict the claim. Higher scores indicate more frequent contradiction, which signals hallucination:

where:

• $n$: number of independently sampled passages
• $G_j$: the $j$-th sampled passage, generated independently from $G$
• $p_{\text{contradict}}(G_j, c_i)$: probability that $G_j$ contradicts claim $c_i$ according to an NLI model
• A high score means the claim is frequently contradicted by other outputs, suggesting hallucination

BERTScore-based consistency measures semantic similarity between the claim and sampled passages, treating low similarity as a signal of inconsistency:

where:

• $\text{BERTScore}(c_i, G_j)$: the BERTScore F1 between the claim and the sampled passage, ranging from 0 to 1
• Subtracting from 1 converts the similarity score into an inconsistency score: low similarity becomes high inconsistency
• A high score means the claim is semantically dissimilar from the sampled passages, suggesting the sampled outputs do not confirm the claim

N-gram overlap provides a lighter-weight alternative that requires no additional models. For each sampled passage $G_j$, the method computes the proportion of n-grams in the claim $c_i$ that also appear in $G_j$, then averages across all samples. The inconsistency score is 1 minus this overlap, so claims whose exact wording or phrasing rarely appears in sampled passages score higher. This approach trades some accuracy for simplicity: it cannot reason about meaning, so paraphrases of the same fact will be treated as dissimilar even if they express identical information. But its speed advantage is significant when the primary bottleneck is inference cost rather than detection precision.

The three variants span a spectrum of computational cost and accuracy. NLI-based scoring is the most accurate but requires running an NLI model for every (sampled passage, claim) pair, which is $n$ NLI calls per sentence. BERTScore is moderately cheaper and handles semantic paraphrases well. N-gram overlap is fastest and imposes no additional model dependencies. In practice, start with n-gram overlap for rapid prototyping, then upgrade to NLI-based scoring if you observe unacceptable false-negative rates on your validation set.

Self-consistency approaches have a fundamental limitation: they can only detect hallucinations that stem from model uncertainty. If a model has confidently encoded a false belief, it will produce that false belief consistently across all samples, and the consistency check will give it a high score indicating ground truth, when in reality it is a systematic error.

This is not a hypothetical concern. Language models are known to have systematic biases toward plausible-sounding but incorrect information in domains where training data is sparse or skewed. Studies of factuality errors in large language models have found that models confidently produce incorrect birth dates, wrong award years, and mistaken nationality claims for less-famous individuals, precisely because the pretraining data contained inconsistent information that the model averaged over in a systematically biased way. Consistency checks will not catch these cases, and may even inflate their apparent reliability.

Additionally, self-consistency requires multiple forward passes through the model. For a passage of $n$ sentences checked against $k$ samples, you need $k + 1$ full generations. At $k = 10$ samples (a common setting in the original SelfCheckGPT paper), this is an 11x increase in generation cost, not counting the NLI scoring step. For large models or high-throughput applications, this cost is often prohibitive.

There is also a subtle failure mode related to topic breadth. When the prompt is broad and the model can respond in many valid ways, the samples will naturally vary in which facts they mention, not because any of them are hallucinating but simply because the space of relevant true facts is large. A sentence about an obscure historical event might receive a high inconsistency score simply because other samples chose to discuss different aspects of the same event. Calibrating consistency thresholds on this kind of prompt is difficult without domain-specific labeled data.