Hallucination Causes › Code Implementation
Language AI HandbookAll chapters Hallucination Causes › Code ImplementationOne section · Sections in this chapter7 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
We will implement three demonstrations: a training data quality analyzer showing how factual accuracy varies across source domains, a visual simulation of exposure bias divergence showing how errors accumulate, and an entity frequency analysis illustrating the tail entity hallucination pattern.
Setup and Imports
import warnings
warnings.filterwarnings("ignore")Simulating Training Data Quality Distribution
We model the quality of training data as a function of domain and source type, showing the known composition of large web corpora.
# Simulate domain-level factual accuracy in training corpora
# Values approximate findings from web-scraping quality studies
domains = [
"Wikipedia",
"Academic Papers",
"News Sites",
"General Blogs",
"Forum Posts",
"Social Media",
"SEO Content",
"AI-Generated Text",
]
# Estimated factual accuracy rates (from 0 to 1)
accuracy_rates = [0.95, 0.88, 0.82, 0.65, 0.58, 0.50, 0.35, 0.45]
# Approximate fraction of typical web corpus from each domain
corpus_fractions = [0.04, 0.03, 0.08, 0.18, 0.15, 0.12, 0.22, 0.18]
# Weighted average accuracy of the full corpus
weighted_accuracy = sum(
acc * frac for acc, frac in zip(accuracy_rates, corpus_fractions)
)Domain factual accuracy breakdown: Domain Accuracy Corpus % ----------------------------------------------- Wikipedia 95% 4% Academic Papers 88% 3% News Sites 82% 8% General Blogs 65% 18% Forum Posts 58% 15% Social Media 50% 12% SEO Content 35% 22% AI-Generated Text 45% 18% ----------------------------------------------- Weighted corpus accuracy 55.20%
Even at a corpus level, the weighted factual accuracy of typical web training data is well below 100%. The model does not distinguish between accurate and inaccurate sources: it processes all tokens under the same next-token prediction objective, learning from errors and truths equally. The dominance of low-accuracy source types (SEO content, social media) by volume is what drives the weighted average down despite the high accuracy of curated sources like Wikipedia and academic papers.
Visualizing Data Quality by Domain
The visualization reveals the core tension in web-scale training: the most accurate sources (Wikipedia at 95%, academic papers at 88%) collectively contribute only about 7% of the corpus. Meanwhile, lower-accuracy sources (SEO content at 35%, general blogs at 65%) make up nearly 40% of the total token volume. The model sees far more noise than signal by volume, even if high-quality sources are given proportional weighting.
Modeling Exposure Bias Error Accumulation
We simulate how a single early error propagates across a generation sequence, modeling the divergence between the true distribution and the model's conditioned distribution under different initial error magnitudes.
import numpy as np
def simulate_exposure_bias_divergence(
error_magnitude: float,
sequence_length: int,
recovery_rate: float = 0.05,
) -> np.ndarray:
"""
Simulate KL divergence accumulation after an error at step 1.
Parameters
----------
error_magnitude : float
Initial divergence introduced by the error (in nats).
sequence_length : int
Number of generation steps to simulate.
recovery_rate : float
Per-step probability that context corrects the error.
"""
divergence = np.zeros(sequence_length)
divergence[0] = error_magnitude
for t in range(1, sequence_length):
# Divergence compounds as subsequent tokens condition on error context
compounding = divergence[t - 1] * (
1.0 + 0.08 * (1 - divergence[t - 1] / 5)
)
# Weak recovery signal from correct context elements
recovery = divergence[t - 1] * recovery_rate
divergence[t] = max(0, compounding - recovery)
# Cap at a maximum divergence (model cannot diverge arbitrarily far
# from training distribution)
divergence[t] = min(divergence[t], 4.5)
return divergence
# Simulate different error magnitudes
steps = 50
error_scenarios = {
"Small error (0.2 nats)": simulate_exposure_bias_divergence(0.2, steps),
"Medium error (0.8 nats)": simulate_exposure_bias_divergence(0.8, steps),
"Large error (1.5 nats)": simulate_exposure_bias_divergence(1.5, steps),
"No error (baseline)": np.zeros(steps),
}
The divergence curves illustrate a key property of exposure bias: errors are self-reinforcing. Once the model commits to an incorrect context, each subsequent token conditions on that error, generating more tokens that are consistent with the error rather than with the ground truth. The small error scenario shows gradual divergence that never fully resolves: even a tiny initial inaccuracy propagates forward without correction. The large error scenario reaches the maximum divergence ceiling quickly, indicating complete loss of calibration relative to the correct distribution.
Entity Frequency and Hallucination Rate
The following analysis models how entity frequency in training data relates to hallucination rates, based on patterns observed in memorization research.
# Model hallucination rate as a function of training frequency
# Based on power-law generalization from entity memorization studies
def hallucination_rate(
frequency: np.ndarray, saturation_freq: int = 500
) -> np.ndarray:
"""
Model hallucination rate as a decreasing function of training frequency.
Parameters
----------
frequency : np.ndarray
Number of times entity appears in training data.
saturation_freq : int
Frequency above which hallucination rate saturates (near zero).
"""
rate = np.exp(-frequency / saturation_freq)
# Add floor: even well-known entities hallucinate occasionally
return 0.03 + 0.92 * rate
# Frequency range from very rare (1 occurrence) to common (10,000 occurrences)
frequencies = np.logspace(0, 4, 500)
hall_rates = hallucination_rate(frequencies)
# Mark key thresholds
threshold_rare = 50
threshold_common = 500
rate_at_rare = hallucination_rate(np.array([threshold_rare]))[0]
rate_at_common = hallucination_rate(np.array([threshold_common]))[0]Hallucination rate model: 1 occurrence -> 95% hallucination rate 50 occurrences -> 86% hallucination rate 500 occurrences -> 37% hallucination rate 10,000 occurrences -> 3% hallucination rate
The steep initial decline of the curve is its most important feature. The drop from 95% hallucination at 1 occurrence to roughly 60% at 50 occurrences is enormous, and it illustrates how desperately the model needs repeated exposures to build reliable knowledge. The curve flattens significantly above 500 occurrences, where the model has enough signal to learn the entity's facts reliably. The 3% floor at high frequencies represents irreducible hallucination from other causes: exposure bias, completion pressure, and the occasional conflicting signal in training data.
Visualizing Completion Pressure
We simulate how completion pressure interacts with knowledge availability, showing the choices a model faces when generating answers to different types of factual questions.
# Simulate model response confidence vs. knowledge availability
# for different question types
question_categories = [
"Famous person biography",
"Major historical event",
"Scientific definition",
"Recent news (post-cutoff)",
"Obscure historical figure",
"Niche scientific finding",
"Local business details",
"Specific legal statute",
]
# Model knowledge availability (how well the model knows the topic)
knowledge_availability = [0.92, 0.88, 0.85, 0.10, 0.25, 0.30, 0.12, 0.20]
# Model output confidence (how confidently it responds)
output_confidence = [0.90, 0.87, 0.88, 0.75, 0.70, 0.72, 0.65, 0.68]
# Expected accuracy (correlated with knowledge, not confidence)
expected_accuracy = [k * 0.95 for k in knowledge_availability]
The gap between knowledge availability and output confidence is the signature of completion pressure. For well-known topics (famous biographies, major events, scientific definitions), the two scores track closely. For low-knowledge topics (post-cutoff news, obscure figures, local businesses, specific legal statutes), confidence remains at 65-75% while knowledge availability drops to 10-30%, creating exactly the conditions for confident hallucination. The model does not know that it does not know; it generates at the same confidence level regardless.
Key Decoding Parameters
The key parameters that affect generation pressure in practice are:
- temperature: Scales the logit distribution before sampling. Higher values () flatten the distribution, increasing randomness and hallucination risk. Lower values () sharpen it toward the most probable tokens, which may reinforce confident errors.
- top_p (nucleus sampling threshold): Controls the fraction of probability mass considered at each step. Higher values increase diversity at the cost of factual consistency.
- top_k: Limits the number of candidate tokens at each step. Lower values reduce but do not eliminate hallucination risk.
- repetition_penalty: Penalizes tokens that appeared earlier in the sequence. Can prevent repetitive loops but may also discourage the model from correctly reusing technical terms.
- max_new_tokens: Controls sequence length. Longer generations face higher cumulative exposure bias divergence, increasing hallucination risk as the sequence grows.