Uncertainty Quantification › Uncertainty Communication

Measuring uncertainty is only half the problem. The other half is communicating it to users in a way that influences behavior. This is where uncertainty quantification meets human factors, interface design, and cognitive science. A technically accurate uncertainty estimate that is poorly communicated might as well not exist: users will either ignore it, misinterpret it, or find it paralyzing.

Humans are poor at reasoning about probabilities in the abstract. If you tell a user that a model response has "72% confidence", most users won't know what to do with that number. Is 72% high? Low? Does it mean they should verify or just proceed? Research on probability communication consistently shows that numerical probabilities are difficult to interpret and often misread. People tend to treat probabilities above 50% as "yes" and below 50% as "no," discarding the continuous information the number contains.

Communication also has asymmetric consequences. If you communicate too much uncertainty, users stop trusting the system even when it's correct, leading to over-verification that wastes time and undermines the value of the AI system. If you communicate too little, users over-rely on incorrect answers, with potentially serious downstream consequences. Getting the calibration of communication right matters for both accuracy and user trust.

There is also a framing effect in how uncertainty is communicated. "There is a 30% chance this is wrong" and "There is a 70% chance this is right" are mathematically identical, but users perceive them very differently. The first frames the response as mostly reliable; the second frames it as risky. Which framing is appropriate depends on the stakes: for a low-stakes recommendation, 70% right is good enough to act on. For a medical diagnosis, 30% wrong is alarming enough to require verification.

Standardized verbal uncertainty scales map numeric probabilities to natural language phrases. The IPCC (Intergovernmental Panel on Climate Change) uses a well-known example for communicating scientific findings:

The key insight is that agreed-upon, consistent mappings are more communicable than ad-hoc hedges. If your system always uses "I'm fairly confident" to mean 70-80%, users can calibrate to that phrase over time. Inconsistent hedging teaches users nothing and prevents them from building accurate mental models of the system's reliability.

Verbal scales have well-known limitations. Different cultures and individuals bring different prior interpretations to words like "likely" or "probable." Research has found that "probable" is interpreted as meaning anywhere from 55% to 85% across different populations. This spread means that verbal scales introduce noise even when the underlying probability estimate is accurate. The IPCC addresses this by publishing explicit numeric mappings and referring readers to them, a practice that AI systems serving technical users might adopt.

Beyond verbal language, interfaces can communicate uncertainty through visual design:

Confidence indicators. A visual bar, color coding (green/yellow/red), or star rating system adjacent to a response provides a quick uncertainty signal without interrupting the flow of text. Color is the most immediately readable but requires careful design for colorblind users.

Expandable uncertainty details. "Click to see why I'm unsure" keeps the primary response clean while giving power users access to detailed uncertainty information such as which specific claims are uncertain or what sources are available.

Source display. Showing which sources support a claim implicitly communicates confidence: more high-quality sources means higher confidence. A response backed by three authoritative citations reads as more reliable than the same content without citations, even if the user doesn't check the citations.

Differentiated formatting. Some systems use italic or muted text for uncertain claims within a response, drawing attention to which specific statements are shaky. This is particularly useful for responses that mix high-confidence background information with uncertain specific claims.

Explicit uncertainty sections. For longer responses, a dedicated "What I'm not sure about" paragraph at the end surfaces all uncertainties together. This is less granular than inline markers but more readable for users who want a high-level reliability assessment before reading the full response.

Natural language contains a rich vocabulary of epistemic markers: words and phrases that signal a speaker's confidence level. In English:

High confidence markers include: "is," "definitely," "certainly," "I'm sure that," "it's clear that."

Medium confidence markers include: "appears to be," "seems likely," "I believe," "probably," "generally speaking."

Low confidence markers include: "might be," "could be," "I'm not certain but," "I may be wrong," "possibly."

Uncertainty markers often include: "I don't know," "I'm not sure," "this is outside my knowledge," "you should verify this."

Well-designed LLM systems calibrate their use of these markers to the actual confidence level of the response. Miscalibrated systems use high-confidence language for uncertain responses, or hedge unnecessarily on confident answers, both of which erode trust over time.

The challenge is that epistemic markers in natural language are not universally interpreted. Different readers bring different calibration priors. "Probably" might mean 60% to one reader and 90% to another. This is precisely the problem that motivated the development of standardized verbal probability scales: by agreeing in advance that "likely" maps to 66-100%, a consistent communication channel is established.

Research in psycholinguistics shows that epistemic markers also interact with domain context. The phrase "I believe" in an academic paper signals a stronger degree of confidence than the same phrase in casual conversation. An AI system serving multiple contexts (customer support, scientific writing, medical triage) would ideally calibrate its hedging vocabulary to the norms of each domain. A medical AI saying "it is probable that this is benign" carries different weight than a shopping assistant saying the same phrase about a product recommendation.

Another subtlety is the difference between hedging about facts and hedging about opinions. "I think this policy is misguided" is an opinion hedge, not a factual uncertainty marker. Well-designed systems distinguish between "I am uncertain whether X is true" and "X is a matter on which reasonable people disagree." Conflating the two misrepresents the nature of the uncertainty and can mislead users about whether verification is useful.

Useful uncertainty communication tells users what to do, not just how uncertain the model is. Raw uncertainty numbers do not guide action; uncertainty combined with recourse does. This means designing uncertainty communication around four principles:

Specificity. "I'm not sure about this" is less useful than "I'm not sure about the exact year; the range is probably 1880-1900." Specific uncertainty helps users decide exactly what to check.

Recourse. "You should verify this with a current source" is more actionable than "I might be wrong." Recourse tells users what action to take, not just that action is needed.

Differentiation. "This fact is contested among historians" helps the user more than a blanket confidence percentage because it explains the nature of the uncertainty and suggests who to consult.

Scope. "I'm confident about the general trend but not the specific numbers" lets the user act on the high-confidence part while verifying the uncertain part. Partial confidence is still useful if it's communicated with scope.

The design goal is trust calibration: users should trust the model's outputs to the degree they deserve trust, neither more nor less. Achieving this requires both accurate uncertainty quantification (measuring the right thing) and effective uncertainty communication (conveying it clearly).

How uncertainty should be handled depends heavily on the downstream task and the stakes involved:

High-stakes decisions. In medical, legal, or financial contexts, any uncertainty above a threshold should trigger human review. The uncertainty score becomes a routing signal: below threshold, deliver the answer; above threshold, escalate to a human reviewer. This architectural pattern converts the uncertainty estimate from a display concern into an operational routing decision.

Retrieval augmentation. Low confidence can trigger a retrieval step. The model's uncertainty about a factual question prompts a search, and the retrieved context enables a more confident and accurate answer. Retrieval-augmented generation (RAG) systems often use this pattern: generate first, measure confidence, retrieve and regenerate if confidence is low.

Interactive clarification. In dialogue systems, uncertainty can trigger a clarifying question: "I'm not sure which John Smith you mean. Can you give me more context?" This is more useful than delivering an uncertain answer because it involves the user in resolving the ambiguity rather than leaving them to discover the error themselves.

Ensemble outputs. When uncertainty is high, present multiple candidate answers with their confidence levels rather than picking one. This is common in information retrieval and recommendation systems where giving users a ranked list is more appropriate than committing to a single recommendation.

The common thread across all these patterns is that uncertainty is treated as an actionable signal, more than a display quantity. Systems that use uncertainty to route, retrieve, or clarify achieve better end-to-end accuracy than systems that generate a single confident answer and hope for the best.