Uncertainty Quantification › Summary

Uncertainty quantification gives language models a way to signal the limits of their knowledge, and gives users a way to know when to trust model outputs. The key ideas from this chapter:

• Calibration measures whether confidence scores align with actual accuracy. A perfectly calibrated model that says "80% confident" is right 80% of the time. Most LLMs are systematically overconfident due to training objectives, RLHF, and temperature effects during inference.

• Temperature scaling is the simplest and most effective post-hoc calibration method. A single learned temperature parameter scales logits to reduce overconfidence. Expected Calibration Error (ECE) quantifies how well-calibrated a model is by measuring the weighted average gap between confidence and accuracy across bins.

• Verbalized uncertainty lets models express confidence in natural language. This is more user-friendly than numeric scores but risks sycophantic hedging, where models learn to hedge for stylistic reasons rather than actual uncertainty. You cannot rely on verbal hedges as direct evidence of model uncertainty without also verifying through behavioral methods.

• Sampling-based methods estimate uncertainty by running the model multiple times and measuring consistency. Self-consistency uses majority voting; semantic entropy groups responses by meaning and computes entropy over semantic clusters. High semantic entropy indicates substantial uncertainty: the model produces diverse and semantically inconsistent answers when it doesn't know the right one.

• Prompt perturbation probes consistency by paraphrasing the same question. A model with solid knowledge answers consistently regardless of phrasing. A model with shallow or inconsistent knowledge is sensitive to surface-form variation, revealing uncertainty that token probabilities might miss.

• Uncertainty communication is as important as uncertainty measurement. Verbal scales, visual indicators, and actionable recourse design help users calibrate their trust appropriately. The goal is to neither over-trust nor under-trust model outputs, and to give users enough information to take the right action when the model is uncertain.

Together, these techniques form a practical toolkit for building language AI systems that are honest about the boundaries of their knowledge, a prerequisite for responsible deployment in any high-stakes context.