Hallucination Detection: NLI, Self-Consistency & Learned Models › Knowledge Base Verification

Entailment-based detection works when you have a reference document to check against. But what if the model is making claims about the world at large, not about a specific source? In an open-domain question-answering system, users ask about historical events, scientific facts, and biographical details for which there is no single "source document" to use as a premise. In that case, you need a different approach: verifying claims against a structured knowledge base that encodes general world knowledge.

Knowledge base (KB) verification is conceptually the most direct approach to hallucination detection. Rather than asking whether a claim is supported by some available text, it asks whether the claim is true according to a structured database of known facts. This is a much stronger standard. It can catch hallucinations that happen to be consistent with vague or ambiguous source documents, and it can verify claims about facts that have never been mentioned in any retrieved context.

The general pipeline involves three steps. First, you parse the generated text into discrete, verifiable atomic claims. Second, you map the mentions in those claims to canonical entities in the knowledge base. Third, you check whether the claims hold according to the knowledge base records. Each step is a non-trivial NLP problem in its own right, and errors compound across steps.

Knowledge bases like Wikidata store information as subject-predicate-object triples. For example, the fact that Marie Curie received the Nobel Prize in Physics is stored as:

$$(\text{Marie Curie},\ \text{received award},\ \text{Nobel Prize in Physics})$$

where:

• The first element is the subject: the entity the claim is about
• The second element is the predicate: the relation or property being asserted
• The third element is the object: the value of that property for the subject

To verify claims, you need to convert natural language claims into the same triple format. A claim extractor maps a sentence $c$ to a set of triples $T(c) = \{(s_k, p_k, o_k)\}_{k=1}^{K}$, where:

• $c$: the natural language claim to be verified
• $K$: the number of distinct factual assertions in the claim
• $(s_k, p_k, o_k)$: the $k$-th extracted triple, representing one factual assertion

This extraction step is itself a hard NLP problem. Common approaches include information extraction pipelines that use rule-based or neural models to identify entity mentions and their relations, LLM-based extraction that prompts a language model to enumerate the factual claims in a sentence in triple form, and Open IE systems that extract triples without a predefined ontology.

LLM-based extraction has become increasingly popular because it handles complex sentences and implicit relations more gracefully than rule-based systems. A simple prompt like "List all factual claims in the following sentence as subject-predicate-object triples" can reliably decompose sentences into atomic claims, especially for well-formed biographical or encyclopedic text. The downside is that LLM-based extraction introduces another model into the pipeline, adding latency and cost.

Each extracted triple $(s, p, o)$ is then verified by querying the knowledge base for whether the triple holds. For Wikidata, this means translating the triple into a SPARQL query and executing it against the Wikidata SPARQL endpoint.

Before querying, entity mentions must be resolved to canonical identifiers. "Curie" in one sentence and "Marie Curie" in another both need to map to the same Wikidata entity (Q7186). Without this normalization, a query for the person "Curie" will find nothing, and the triple will be incorrectly flagged as unverifiable.

The disambiguation problem is more subtle than it first appears. Consider the sentence "Jordan played for the Bulls." Without context, "Jordan" could refer to Michael Jordan the basketball player or the country of Jordan, but the phrase "played for the Bulls" strongly implies the basketball player. A naive entity linker that matches based solely on the most common entity for a surface form will fail on ambiguous cases. The context of the surrounding text, particularly other named entities and the semantic field of the sentence, is essential for correct disambiguation.

Learned entity linkers address this using a two-stage approach that has become the dominant paradigm. In the first stage, a bi-encoder embeds the mention in context and retrieves the top-K candidate entities from a dense index using approximate nearest-neighbor search. The mention encoder processes the surface form plus surrounding context, while the entity encoder processes the entity's name and description from the knowledge base. The top-K candidates are retrieved by computing the dot product between the mention embedding and all entity embeddings. In the second stage, a cross-encoder re-ranks the candidates by jointly encoding the mention context and the entity description. This two-stage design is efficient enough for real-time systems while achieving high disambiguation accuracy on standard benchmarks.

For claims about highly specific or niche entities, entity linking often fails entirely because the entity simply does not exist in the knowledge base's entity catalog. A language model generating text about a newly formed startup, a novel scientific concept, or a person who is not famous enough to have a Wikidata entry will produce entity mentions that no linker can resolve. This coverage limitation is one of the fundamental constraints of the KB verification approach and is why it works best for well-known entities in stable domains.

Once triples are extracted and entities are linked, you query a knowledge base to check consistency. For Wikidata, this means using SPARQL queries to retrieve the known relations for a given entity pair. If the expected predicate exists with the expected object, the claim is verified; if not, the claim is flagged as potentially false.

The limitation of this approach is coverage. Wikidata contains roughly 100 million triples, which sounds large but is a tiny fraction of all human knowledge. Claims about niche topics, recent events, or specialized domains often have no relevant entries. In those cases, the verifier must either abstain (producing no score) or fall back to a different approach.

A critical failure mode is the temporal gap. Wikidata reflects the state of the world as of when entries were last updated, which means verifying claims about recent events, people, or discoveries will frequently return no match, even when the claim is true. This issue is particularly pronounced for rapidly changing domains like sports statistics, corporate leadership, and ongoing scientific research. A claim that "Dr. X is the current CEO of Company Y" may be true but unverifiable if Wikidata's entry for Company Y has not been updated recently. Treating unverifiable claims as hallucinations inflates the false positive rate in temporally sensitive applications.

Knowledge base verification also struggles with relational claims that involve implicit temporal qualifications. "Einstein was a professor at Princeton" is technically true for a specific period of his life, but a KB query might find a different affiliation if the record was indexed at a different time. Properly handling temporal predicates requires additional logic to check the validity period of each triple, which most systems do not implement.

A hybrid approach that has gained significant traction is FActScore (Factual Precision Score). Rather than using a structured knowledge base, FActScore retrieves relevant passages from Wikipedia using a dense retrieval model, then checks each atomic fact against the retrieved passages using an NLI model. This approach captures the benefits of KB verification (checking against a broad knowledge source) while avoiding the coverage and temporal limitations of structured databases.

The key design insight is that Wikipedia contains far more information than any structured KB, covers a much wider range of entities, and is updated more frequently. Dense retrieval allows the system to find the most relevant passages for any given claim, even for niche topics not covered by Wikidata. The NLI step then checks whether those passages support or contradict the claim, using the same entailment reasoning we saw in the previous section.

The process for a generated biography or factual response $G$:

1. Decompose $G$ into atomic facts $\{f_1, f_2, \ldots, f_n\}$ using an LLM prompted to enumerate one claim per fact
2. For each fact $f_i$, retrieve relevant Wikipedia passages using a retrieval model such as DPR or BM25
3. Score each fact as supported or unsupported using an NLI model applied to the retrieved passages
4. Report the proportion of supported facts:

$$\text{FActScore}(G) = \frac{|\{f_i : \text{supported}(f_i) = 1\}|}{n}$$

where:

• $n$: total number of atomic facts extracted from the generated text
• $\text{supported}(f_i) = 1$: indicates that fact $f_i$ is verified as supported by retrieved passages
• $|\cdot|$: cardinality (count) of the set of supported facts
• $\text{FActScore}(G) \in [0, 1]$: proportion of facts that are supported; 1.0 means all facts are grounded, 0.0 means none are

FActScore was proposed alongside a large benchmark of human-written factuality annotations for LLM-generated biographies, making it one of the most carefully evaluated detection methods available. The benchmark revealed that popular LLMs at the time of publication had FActScore values between 0.3 and 0.6 on biographical text, meaning between 40% and 70% of their factual claims were not verifiable against Wikipedia. This quantification was influential in motivating investment in hallucination mitigation research.

The main cost of FActScore is its pipeline complexity. Each fact requires a retrieval call, which involves encoding the query with a dense retrieval model and running approximate nearest-neighbor search over a large index. Then each retrieved passage must be scored with an NLI model. For a single paragraph with a dozen facts, this results in dozens of model calls. The atomic fact decomposition step itself requires an LLM call. Total inference cost can be 50 to 100 times higher than entailment-only scoring against a fixed source document.