Uncertainty Quantification › Sampling-Based Uncertainty

Instead of trying to extract a confidence score from a single model run, sampling-based methods estimate uncertainty by running the model multiple times on the same query and measuring how much the outputs vary. High variance across samples suggests the model is uncertain; low variance suggests confidence.

This approach sidesteps several problems with token-level probability as a confidence measure. Token probabilities are affected by generation artifacts (temperature, sampling parameters), training biases, and RLHF shifts that don't necessarily reflect factual uncertainty. Behavioral variance across samples is a more direct measure of whether the model would give consistent answers in deployment: a model that gives the same answer ten times out of ten when sampled is behaving consistently, regardless of what probabilities it assigns internally.

The basic idea is Monte Carlo estimation: draw $N$ samples from the model's distribution and compute a statistic over them. The variance of that statistic estimates uncertainty.

For a yes/no question, this is straightforward:

1. Generate $N$ responses with temperature $T > 0$ (to introduce sampling randomness)
2. Count how many responses say "yes" versus "no"
3. Let $p$ be the fraction of "yes" responses: $p = |\{i : y_i = \text{"yes"}\}| / N$
4. Compute binary entropy as the uncertainty measure:

$$H(p) = -p \log p - (1-p) \log(1-p)$$

where:

• $p$ is the proportion of responses in the majority class
• $H(p)$ reaches its maximum of $\log 2 \approx 0.693$ nats (or 1 bit) when $p = 0.5$, indicating maximum uncertainty
• $H(p) = 0$ when $p \in \{0, 1\}$, indicating complete agreement and minimum uncertainty

Higher entropy (answers split near 50/50) means higher uncertainty. Near-unanimous answers indicate confidence. The beauty of this approach is that it requires no access to model internals: you only need to observe the model's outputs.

The computational cost is the main drawback. Running $N$ samples means $N$ forward passes through the model. For a model with a single generation costing $0.01 and $N = 10$ samples per query, the uncertainty estimation costs $0.10 per query, ten times more than a single generation. For high-throughput production systems, this cost is often prohibitive.

For open-ended responses, simple string comparison is too strict. "Paris is the capital of France" and "France's capital city is Paris" express the same fact but would be counted as different answers under exact match. Semantic consistency measures whether multiple responses convey the same meaning, independent of surface form.

One approach: use an NLI (natural language inference) model to check whether pairs of sampled responses entail each other. If response $A$ entails $B$ and $B$ entails $A$, they are semantically equivalent. The fraction of response pairs that are semantically equivalent is the consistency score. Low consistency means high uncertainty.

$$\text{Consistency}(y_1, \ldots, y_N) = \frac{1}{\binom{N}{2}} \sum_{i < j} \mathbb{1}[\text{NLI}(y_i, y_j) = \text{entail} \;\land\; \text{NLI}(y_j, y_i) = \text{entail}]$$

where $\text{NLI}(a, b) = \text{entail}$ means "response $a$ entails response $b$". The bidirectional entailment check (both $A$ entails $B$ and $B$ entails $A$) is a proxy for semantic equivalence. NLI models are fast compared to generation, so the consistency computation adds minimal overhead once samples are available.

The limitation is that NLI models are themselves imperfect and can make errors on semantically subtle cases. Responses that are paraphrases of the same claim may fail the entailment check if they use different vocabulary. This is an active area of improvement in the field.

Self-consistency (Wang et al., 2022) was originally proposed to improve reasoning accuracy but also provides a natural uncertainty estimate. The method generates multiple chain-of-thought reasoning paths and uses majority voting to select the final answer.

The process works as follows. The model is prompted to think step by step, and this prompt is sent $N$ times with non-zero temperature to produce $N$ different reasoning paths. Each path arrives at a final answer. Majority voting selects the most common final answer as the output. The fraction of samples agreeing with the majority answer is a confidence score. If 9 out of 10 sampled paths arrive at the same answer, confidence is 0.9. If only 6 out of 10 agree, confidence is 0.6.

Self-consistency works well for structured tasks (math problems, multiple-choice QA) where you can extract a clean final answer from each reasoning path. For open-ended generation, you need semantic clustering instead of exact match voting: run all sampled responses through an NLI model to cluster them by meaning, then take the fraction belonging to the largest cluster as the confidence estimate.

The insight behind self-consistency is that different reasoning paths through a problem serve as approximate independent samples from the model's belief distribution. When the model reliably knows the answer, most paths converge on it. When the model is uncertain, paths diverge into different regions of the answer space. The variance across paths encodes the model's behavioral uncertainty.

Semantic entropy (Kuhn et al., 2023) formalizes sampling-based uncertainty using information theory. Rather than computing entropy over raw text samples (which conflates surface variation with semantic variation), it computes entropy over semantic clusters.

The procedure has four steps:

1. Sample $N$ responses $y_1, \ldots, y_N$ from the model with non-zero temperature
2. Group responses into semantic equivalence classes $C_1, C_2, \ldots, C_K$ using bidirectional NLI entailment
3. Estimate the probability of each class: $P(C_k) \approx |C_k| / N$
4. Compute semantic entropy:

$$\text{SE} = -\sum_{k=1}^{K} P(C_k) \log P(C_k)$$

where $K$ is the number of distinct semantic clusters, $C_k$ is the $k$-th cluster, $|C_k|$ is the number of responses assigned to it, and $N$ is the total number of sampled responses. The $\log$ here is the natural logarithm, so SE is in units of nats.

Semantic entropy is zero when all responses are semantically identical (certain) and increases as responses become more diverse in meaning. It is invariant to surface-form variation: if all responses say the same thing in different words, they form one cluster and SE is zero. It correlates more strongly with factual accuracy than token-level probability entropy, which is noisier because it treats surface variation as signal.

In neural networks, Monte Carlo Dropout approximates Bayesian inference by keeping dropout active at test time. Multiple forward passes through the network, each with a different random dropout mask, produce a distribution over predictions. The variance of this distribution estimates epistemic uncertainty.

The theoretical foundation is that a neural network with dropout at every layer is mathematically equivalent to a deep Gaussian process under certain conditions (Gal and Ghahramani, 2016). Each forward pass with a different dropout mask can be interpreted as sampling from an approximate posterior distribution over the network's weights. The variance across these samples therefore estimates the posterior uncertainty about the model's predictions.

For language models, this has limited practical use because large language models typically do not use dropout during inference, and enabling it changes the model's behavior unpredictably. The sheer scale of modern LLMs makes the Gaussian process approximation less precise, and activating dropout at inference time often degrades output quality substantially. This makes it difficult to separate "uncertainty due to knowledge limits" from "degradation due to random dropped activations."

But the conceptual framework carries over: any source of stochasticity (temperature sampling, different random seeds, different prompt phrasings) can be used to generate a distribution from which uncertainty is estimated. Temperature sampling is the most direct analogue. It introduces randomness proportional to the model's own uncertainty in its token predictions, and sampling multiple times with non-zero temperature produces a distribution over outputs that encodes the model's behavioral uncertainty about what to say.

A related technique uses paraphrased versions of the same question to probe consistency. If the model gives different answers to "What year was Einstein born?" and "When was Einstein born?", the inconsistency suggests uncertainty or knowledge gaps. Formally:

1. Generate $K$ semantically equivalent paraphrases of the question
2. Obtain the model's answer to each paraphrase
3. Measure semantic consistency across answers
4. Treat inconsistency as a proxy for uncertainty

This approach is robust to the specific sampling temperature and doesn't require running the model multiple times on the exact same input (which can be dominated by sampling artifacts from a single phrasing).

The intuition is that a model with stable knowledge should retrieve consistent answers regardless of how a question is phrased. When knowledge is solid, the retrieval process is robust to surface-form variation. When it is shaky, small changes in phrasing tip the model toward different answers, because the model is pattern-matching against surface forms rather than retrieving a stable fact. Sensitivity to question phrasing is a symptom of shallow or inconsistent encoding.

Prompt perturbation is especially useful for detecting factual inconsistency: the model might answer "Einstein was born in 1879" when asked directly, but answer "Einstein was born in Ulm in 1878" when a slightly different phrasing activates a different context window. Neither answer carries explicit uncertainty; they contradict each other. Standard log-probability methods might rate both answers as high-confidence because the model generates each fluently. Only consistency checking reveals the problem.

The practical challenge is generating high-quality paraphrases automatically. Large models can be prompted to paraphrase, but this adds latency and cost. Rule-based perturbations (changing word order, substituting synonyms, adding neutral context) are faster but less semantically faithful. Research on this technique has shown that consistency correlates with factual correctness in the 0.6-0.7 range, which is meaningful but not sufficient as a standalone reliability signal.