Uncertainty Quantification › Confidence Calibration

Calibration is the foundational concept in uncertainty quantification. A model is well-calibrated if its stated confidence probabilities match the frequency of being correct. Understanding calibration requires first understanding how confidence is defined for language models, then understanding how that confidence deviates from ideal behavior, and finally seeing how it can be corrected.

Language models output a probability distribution over vocabulary tokens at each decoding step. Given a context $c$ (the input prompt plus all tokens generated so far), the model assigns a probability to each possible next token $t$:

$$P(t \mid c) = \frac{\exp(z_t / T)}{\sum_{v \in V} \exp(z_v / T)}$$

where:

• $z_t$ is the logit (pre-softmax score) for token $t$, computed from the model's final linear layer
• $V$ is the full vocabulary (commonly 32,000 to 128,000 tokens in modern LLMs)
• $T$ is the temperature parameter, defaulting to $T = 1$ at inference time

The logit $z_t$ is a raw score proportional to the model's internal "preference" for token $t$. The softmax function converts these scores into a valid probability distribution by exponentiating and normalizing. The temperature $T$ controls how sharp or flat the resulting distribution is: at $T = 1$ the distribution reflects the raw model preferences, at $T < 1$ it concentrates mass on the top token (more deterministic), and at $T > 1$ it spreads mass more evenly (more random).

This gives us token-level probabilities. For a complete response $y = (t_1, t_2, \ldots, t_n)$, the sequence probability follows from the chain rule of probability, factoring the joint distribution over the autoregressive generation process:

$$P(y \mid x) = \prod_{i=1}^{n} P(t_i \mid x, t_1, \ldots, t_{i-1})$$

where $x$ is the original input prompt. Each factor is the probability of generating the $i$-th token given the prompt and all previously generated tokens.

Sequence probabilities are products of many values below 1, so they shrink exponentially with length. A response of 20 tokens might have a sequence probability of $10^{-30}$, making direct comparison across different-length responses misleading. A short, confident answer would appear more probable than an equally correct but longer one simply because it has fewer multiplicative terms.

The log-probability resolves this by converting the product into a sum:

$$\log P(y \mid x) = \sum_{i=1}^{n} \log P(t_i \mid x, t_1, \ldots, t_{i-1})$$

Since $\log P(t_i \mid \cdot)$ is negative (probabilities are between 0 and 1, so their log is $\leq 0$), the log-probability is a non-positive number: less negative means higher confidence. Length-normalized log-probability divides by $n$ to make scores comparable across responses of different lengths:

$$\text{score}(y \mid x) = \frac{1}{n} \sum_{i=1}^{n} \log P(t_i \mid x, t_1, \ldots, t_{i-1})$$

This normalized score is the most commonly used single-number summary of how "confident" a model is about a particular response. It represents the average per-token log-probability: a value near 0 means the model assigned high probability to each token it generated (confident), while a very negative value means many tokens had low probability (uncertain).

There is one important subtlety here. The length-normalized log-probability measures fluency, not accuracy. A model can fluently generate a confident-sounding false statement with high per-token probability. The score reflects how consistent the output is with the model's training distribution, not how consistent it is with ground truth. This disconnect is the central challenge that all calibration methods attempt to address.

A classifier is perfectly calibrated if, among all predictions made with confidence $p$, exactly a fraction $p$ are correct. Formally:

$$P(\hat{y} = y \mid \hat{p} = p) = p \quad \forall p \in [0, 1]$$

where $\hat{y}$ is the predicted class, $y$ is the true label, and $\hat{p}$ is the model's stated confidence.

You can measure calibration empirically by grouping predictions into bins by confidence value, computing accuracy within each bin, and comparing the two. A reliability diagram (also called a calibration curve) plots predicted confidence on the x-axis against observed accuracy on the y-axis. Perfect calibration is the diagonal line $y = x$: a model predicting 70% confidence should be right 70% of the time.

Reliability diagrams are powerful diagnostic tools. If the curve lies below the diagonal, the model is overconfident: it claims higher confidence than its actual accuracy. If the curve lies above the diagonal, the model is underconfident: its stated confidence is lower than its actual accuracy. The shape of the deviation reveals the type of miscalibration. A model that is overconfident across all confidence levels has a curve uniformly below the diagonal. A model that is overconfident only at high confidence values (but well-calibrated at low confidence) has a curve that tracks the diagonal until about 0.7, then bends below it. These different patterns call for different correction methods.

ECE has a known limitation: if most predictions fall in a few bins, those bins dominate the metric even if the model is well-calibrated in the sparse bins. Adaptive calibration error (ACE) addresses this by using equal-frequency bins rather than equal-width bins, making sure each bin contains roughly the same number of predictions. Maximum calibration error (MCE) reports the worst-case bin gap rather than the average. For most purposes, standard ECE with 10-15 bins provides an adequate first-pass diagnostic.

Most large language models are overconfident: their confidence scores are systematically higher than their actual accuracy. If you ask a model 100 factual questions where it expresses "95% confidence," you might find it correct on only 70. The calibration curve bends below the diagonal throughout the range.

This happens for several distinct reasons that reinforce each other:

Training objective mismatch. The next-token prediction objective optimizes for placing probability mass on the correct token during training. On the training data, the model repeatedly encounters the correct answer and learns to push high probability toward it. But this process doesn't teach the model to reserve lower probabilities for questions it's less certain about. During inference on novel questions, the model still outputs high probabilities because that's what it was optimized to do, even when the answer is uncertain.

RLHF calibration shift. Reinforcement learning from human feedback trains models to produce responses that humans rate highly. Confident, decisive answers systematically receive higher ratings than hedged, qualified ones. Annotators typically find "The Battle of Hastings was fought in 1066" more helpful than "The Battle of Hastings was fought around 1066, though I'm not entirely certain of the exact year." This creates a training signal that directly penalizes verbal uncertainty and rewards overconfidence.

Distribution shift. Models are calibrated on their training distribution but evaluated on a much wider range of queries. When a model encounters a question near the edge of its training knowledge, it doesn't necessarily recognize that it's at the edge, so its confidence doesn't drop accordingly. A model trained primarily on English Wikipedia may be quite confident on topics that happen to be well-covered there, even when its knowledge is shallow or biased by the articles' own errors.

Temperature effects at inference. Many production systems use low inference temperatures (often $T < 1$) to produce more deterministic, less repetitive outputs. Low temperature sharpens the probability distribution, pushing top-token probabilities higher without necessarily improving accuracy. A model running at $T = 0.5$ will report higher token probabilities than the same model at $T = 1.0$ even when generating the same output.

Lack of calibration objectives during training. Standard language model training has no explicit calibration objective. The model is never asked "how often are you right when you say you're 90% confident?" Calibration can only emerge implicitly from the training data, and since most training text doesn't include explicit uncertainty labels, the model has no direct signal to learn from.

Understanding these causes matters because they suggest different remedies. RLHF miscalibration calls for changing the reward function. Temperature effects call for post-hoc scaling. Distribution shift calls for domain-specific calibration. No single method addresses all causes simultaneously.

Temperature scaling is the simplest and most effective post-hoc calibration technique. After training, you learn a single scalar temperature $T^*$ on a held-out calibration set by minimizing the negative log-likelihood:

$$T^* = \arg\min_{T} \; -\sum_{i} \log \frac{\exp(z_{y_i} / T)}{\sum_v \exp(z_v / T)}$$

where:

• $T$ is the temperature parameter (the single scalar we are optimizing)
• $z_{y_i}$ is the logit of the true class for example $i$
• The sum in the denominator is over all vocabulary items $v$

A temperature $T > 1$ softens the distribution (reduces confidence), while $T < 1$ sharpens it. For overconfident models, the optimal $T^* > 1$. The key insight is that the optimization is over just one parameter, making it fast and stable even with small calibration sets.

Temperature scaling is appealing because it modifies only the confidence values, not the ranking of predictions. The most likely class remains the same; only the probability it's assigned changes. This means you can apply temperature scaling to an already-deployed model without changing its output quality, adding a calibration layer on top of the existing system.

The main limitation is that temperature scaling applies the same correction uniformly across all inputs and confidence levels. If a model is overconfident on some question types but underconfident on others, a single temperature can't correct both simultaneously. More flexible methods are needed for heterogeneous miscalibration.

More flexible calibration methods learn non-linear mappings from raw confidence scores to calibrated probabilities.

Platt scaling fits a logistic regression model to the raw scores:

$$\hat{p} = \sigma(A \cdot s + B)$$

where $s$ is the raw confidence score, $\sigma$ is the sigmoid function, and $A$ and $B$ are learned parameters. With two parameters instead of one, Platt scaling can handle asymmetric miscalibration: for example, a model that is overconfident at high scores but underconfident at low scores. The logistic function also ensures the output is a valid probability between 0 and 1.

Isotonic regression fits a piecewise constant monotone function from raw scores to probabilities. The monotonicity constraint preserves the model's original ranking (if score $A >$ score $B$ before calibration, then $P(A) > P(B)$ after), while the piecewise constant shape allows arbitrary flexible correction within each piece. Isotonic regression can correct any form of miscalibration as long as it is monotone, but it requires more calibration data to avoid overfitting. With small calibration sets, the piecewise function may overfit to the specific examples seen during calibration.

Both methods require a labeled calibration set, which is harder to construct for language models than for classifiers. You need questions with known ground-truth answers, model confidence scores, and correctness labels. For factual QA, this can be approximated by sampling questions from a knowledge base and automatically checking answers. For more complex tasks, human annotation is required.

Calibration for open-ended generation is qualitatively harder than for classification. A factual QA task has clear correct answers, but for summarization or open-ended dialogue, "correct" is a spectrum. Two practical approaches have emerged:

Binary correctness framing. Define correctness as "does the response match the ground truth?", use an automated evaluator (such as an NLI model or an LLM-as-judge) to label responses, and treat the problem as binary classification. This lets standard calibration metrics apply directly. The limitation is that the quality of the calibration estimate is bounded by the quality of the automated evaluator. If the evaluator makes systematic errors, those errors propagate into the calibration measurement.

Semantic clustering. Run the model multiple times on the same question, cluster responses by meaning, and treat the frequency of the most common cluster as a proxy for confidence. Well-calibrated models should produce high-frequency clusters when they're correct and more diverse clusters when uncertain. This approach does not require ground-truth answers, making it applicable even when automated evaluation is not possible. We cover this in detail in the sampling-based uncertainty section below.