Uncertainty Quantification › Uncertainty Quantification

Language models generate text with apparent confidence. They produce fluent sentences, cite statistics, name sources, and answer questions in a tone that reads as authoritative. But that surface confidence is not the same as actual reliability. A model can be wrong and sound certain, or correct and sound hesitant. The two things, confidence and accuracy, are entirely independent unless something specifically ties them together.

Uncertainty quantification (UQ) is the discipline of measuring and communicating how much a model knows. In traditional machine learning, this means producing probability estimates that accurately reflect the likelihood of being correct. A classifier that says "90% confident" on 100 examples should be right about 90 of them. That property is called calibration, and well-calibrated models are useful precisely because their confidence scores are actionable. When a model's score is high, you can trust the answer. When it is low, you know to send it for review.

Language models complicate this picture in several ways. First, they output distributions over entire token sequences, not single-class probabilities. A classification model has a small, fixed output space and can directly assign a probability to each class. A language model's output space is astronomically large: every possible sequence of tokens is a valid output, and measuring confidence over that space requires either clever approximations or accepting a necessarily incomplete picture. Second, large language models have been trained through reinforcement learning from human feedback (RLHF), which systematically pushes them toward confident-sounding responses because human raters often prefer decisive, authoritative answers over hedged, qualified ones. This is a direct calibration-corrupting force built into the training process itself. Third, they operate on open-ended tasks where "correctness" is hard to define and harder to measure. The result is that modern large language models are frequently miscalibrated: they express more confidence than their actual accuracy warrants, especially on questions near the edge of their knowledge.

Consider a concrete example. You ask a language model who won the 1987 Nobel Prize in Chemistry. The model answers confidently and fluently. But unless something in its training process specifically tied that confident tone to verified correct answers, the fluency is evidence only of good language modeling, not factual accuracy. The model generates the words "Donald Cram, Jean-Marie Lehn, and Charles Pedersen" with high token probability not because it checked a database but because those tokens follow plausibly from the context in a way that matches its training distribution. Fluency and accuracy are not the same thing, and well-calibrated uncertainty is what bridges the gap between them.

As we explored in the previous chapters on hallucination types, detection, causes, and mitigation, the problem of hallucination is fundamentally about a model producing output that doesn't match reality. Uncertainty quantification provides a complementary lens: instead of asking "did this output hallucinate?", it asks "how confident should we be in this output before we even deliver it?" Properly quantified uncertainty can trigger retrieval augmentation, send answers for human review, or communicate limitations directly to users. The previous chapter on attribution and citation handles the question of "where did this information come from?". Uncertainty quantification handles "how certain is the model that the information is correct?"

This chapter covers the four major pillars of uncertainty quantification for language models:

• Confidence calibration: measuring and correcting the gap between expressed and actual confidence
• Verbalized uncertainty: training models to express their uncertainty in natural language
• Sampling-based uncertainty: estimating uncertainty by observing variance across multiple model outputs
• Uncertainty communication: designing interfaces and language to convey uncertainty to end users effectively