Calibration › Calibration

When you ask a language model a question, it returns an answer and a probability. That probability represents the model's confidence: "I am 90% sure the answer is Paris." The practical question is whether you should believe it. If the model says it is 90% confident one hundred times, should it be correct about ninety of those answers? Calibration is the study of this alignment between stated confidence and observed accuracy. Understanding calibration matters because it determines whether we can trust a model's uncertainty estimates when making decisions. In high-stakes applications such as medical diagnosis, legal advice, or autonomous systems, a model's confidence score directly influences how humans act upon its predictions. If the confidence is misleading, you may either disregard correct predictions due to apparent uncertainty or trust erroneous predictions because of false certainty.

A perfectly calibrated model is one where the confidence scores match the empirical probabilities of correctness. If a weather forecaster predicts a 70% chance of rain every day for a month, and it rains on exactly 70% of those days, that forecaster is well-calibrated. Similarly, if a language model assigns 0.8 probability to its top prediction, and that prediction is correct exactly 80% of the time across many instances, the model is calibrated at that confidence level. This concept extends beyond individual predictions to the entire distribution of the model's outputs. Consistency across the full range of confidence values ensures that the probabilities generated by the model carry meaningful semantic content about uncertainty. A model that is calibrated at every confidence level simultaneously possesses what researchers call "global calibration," meaning there is no systematic bias in how it reports uncertainty anywhere on the probability scale.

Calibration differs from accuracy. A model can achieve high accuracy while being poorly calibrated. It might be correct most of the time but systematically overconfident, declaring 99% certainty even when it is wrong half the time. Conversely, a model can be perfectly calibrated but have low accuracy if it consistently expresses low confidence. As we discussed in Cross-Entropy Loss, the loss function encourages the model to assign high probability to correct answers, but it does not strictly enforce that these probabilities reflect true likelihoods. The training objective is discriminative rather than probabilistic: it rewards the model for ranking the correct answer highest, not for estimating how often it will be right. Modern large language models, despite their impressive performance on benchmarks we will explore in later chapters, often exhibit significant miscalibration, particularly becoming overconfident as scale increases.

To understand why calibration matters practically, consider the difference between a model used as an oracle versus one used as a collaborator. When a model acts as an oracle, you either accept or reject its outputs wholesale. But in most real-world deployments, models serve as collaborators where the confidence score determines how much human oversight is applied. A doctor using an AI diagnostic system might review every prediction where confidence falls below 0.85, and accept those above without further scrutiny. If that threshold was calibrated on a well-calibrated model, this policy works. If the model is overconfident, the doctor is inadvertently accepting many incorrect diagnoses because the model's stated 87% confidence corresponds to only 60% accuracy in practice. The downstream harm of this miscalibration is not just statistical: it translates directly into misallocated human attention and worse patient outcomes.