Hallucination Mitigation: RAG, Decoding, and Training › Uncertainty Expression

All of the strategies above try to prevent the model from generating false content or verify it afterward. Uncertainty expression takes a different approach: rather than preventing hallucination, it asks the model to communicate when it is uncertain, allowing users to calibrate their trust in the output.

This is not a fallback for when prevention fails. Uncertainty expression and hallucination prevention are complementary strategies. Even a well-mitigated system operating on difficult out-of-domain queries will sometimes be uncertain. A system that can recognize and communicate this uncertainty is more useful than one that generates confident answers of unknown reliability.

The simplest form of uncertainty expression is verbalization: training the model to express confidence in natural language. Phrases like "I believe," "I'm not certain, but," "to my knowledge as of my training cutoff," or "you may want to verify this" communicate epistemic status directly to the user.

RLHF can instill this behavior when human annotators reward responses that accurately hedge versus those that confidently assert uncertain information. Models trained this way learn a rough mapping from their internal uncertainty states to verbal hedges. The calibration is imperfect, however: models often express high-confidence hedges ("to the best of my knowledge") even when they are essentially guessing, and sometimes express confident assertions on uncertain facts.

A better-calibrated hedging strategy distinguishes between types of uncertainty:

• Knowledge cutoff uncertainty: facts that may have changed since training ("As of my training data, X, but this may have since changed")
• Low-coverage uncertainty: facts about entities underrepresented in training ("I don't have detailed information about X")
• Contradictory-source uncertainty: facts where training data contained conflicting information ("Sources disagree on this, but a common claim is X")
• Inference uncertainty: conclusions requiring long reasoning chains where errors may accumulate ("This requires several inference steps; I'd recommend verifying")

Teaching models to distinguish these uncertainty types requires training data that explicitly labels which type applies. Generic hedging is easy to learn but less useful; type-specific hedging is harder to train but provides more actionable information to the user.

When uncertainty is high enough, the right response may be no response at all, or an explicit refusal with guidance for the user. "I don't know" is more useful than a confident hallucination, provided the model correctly identifies when it does not know.

This idea is older than LLMs. In statistical machine learning, "abstaining classifiers" or "reject options" are well-studied mechanisms: a classifier can output a prediction or say "I am not confident enough to classify this input." The accuracy on the subset of inputs where the model does predict tends to be much higher than overall accuracy. For language models, the analogous behavior is refusing to answer questions that fall outside the model's reliable knowledge, rather than fabricating a plausible-sounding response.

Teaching models to abstain requires training signal. Instruction-tuned models that have been trained to always attempt an answer have, by construction, been trained away from appropriate refusal. Recalibrating this behavior requires either:

• Refusal examples in SFT data: training examples where the correct output is an explicit "I don't have reliable information about this" rather than an attempted answer
• Reward modeling that values refusal: explicitly rewarding accurate refusals in RLHF, not just rewarding correct answers

The difficulty in constructing refusal examples is knowing when refusal is warranted. A training example that says "the model should refuse to answer X" implicitly encodes a judgment about what the model should and should not know, which requires curators to have their own ground truth. This is tractable for clearly out-of-scope queries (events after the training cutoff, fictional events in real-world framing) but harder for tail-entity queries where the model simply does not have high-quality signal.

The risk is over-refusal: a model too quick to say "I don't know" is less useful than intended. Calibrating the refusal threshold requires measuring both false positive and false negative rates, which requires ground truth for a representative set of questions. A model that refuses 30% of answerable queries to avoid hallucinating on the remaining 10% may be worse in aggregate than a model that hallucinates 10% and answers correctly 90%.

Beyond verbalized uncertainty, models expose quantitative uncertainty through their output probability distributions. At each decoding step, the logits encode how much probability mass the model places on each possible token. Low entropy (high confidence in one token) suggests the model has a clear preference. High entropy (probability spread across many tokens) suggests uncertainty.

The log probability of a complete response is the sum of log probabilities for each token:

$$\log P(y \mid x) = \sum_{t=1}^{|y|} \log P(y_t \mid x, y_{1:t-1})$$

where:

• $y$: the full generated response as a sequence of tokens
• $y_t$: the $t$-th token in the response
• $x$: the input prompt
• $|y|$: the length of the response in tokens

Low overall log probability signals that the model assigned relatively low probability to this particular completion, which can be used as an uncertainty signal. Comparing log probabilities across multiple sampled responses gives a distribution of confidence scores; the variance of this distribution is a measure of generation uncertainty.

Token-level entropy is a more fine-grained signal than whole-sequence probability. Rather than computing the log probability of the entire response, you compute the entropy $H_t = -\sum_{x'} P(x' \mid x, y_{1:t-1}) \log P(x' \mid x, y_{1:t-1})$ at each token position $t$. High-entropy positions indicate tokens where the model is uncertain. This allows you to identify specific tokens or spans that are uncertain rather than flagging the entire response. For named entities and numbers (which are common hallucination sites), token-level entropy tends to spike when the model is confabulating.

Sequence probability is not perfectly calibrated with factual accuracy. Short responses have lower log probabilities than long ones simply because of length. Fluent hallucinations can have high log probabilities. But combined with other signals (diversity across samples, entropy of individual token predictions), probability-based uncertainty estimates provide useful information.

Selective prediction formalizes the abstention idea: a model predicts on inputs where it is confident and abstains on inputs where it is not. The key design choice is the confidence threshold $\tau$: predict when estimated confidence $c(x) \geq \tau$, abstain otherwise.

A useful framework for evaluating selective prediction is the coverage-accuracy tradeoff:

$$\begin{aligned} \text{Coverage}(\tau) &= P(c(x) \geq \tau) \\ \text{Accuracy}(\tau) &= P(\hat{y} = y \mid c(x) \geq \tau) \end{aligned}$$

where:

• $\tau$: the confidence threshold above which the model predicts rather than abstaining
• $c(x)$: the confidence estimate for input $x$ (e.g., max token probability, mean log probability, or self-consistency score)
• $\text{Coverage}(\tau)$: the fraction of inputs on which the model makes a prediction at threshold $\tau$
• $\text{Accuracy}(\tau)$: the fraction of those predictions that are correct
• $\hat{y}$: the model's prediction; $y$: the ground truth

As $\tau$ increases, coverage decreases (fewer inputs are answered) but accuracy on the answered subset increases. Plotting this tradeoff (AUARC, area under the accuracy-rejection curve) gives a single-number summary of how well the confidence estimate separates cases where the model is right from cases where it is wrong.