TruthfulQA › Evaluation Methodology

TruthfulQA employs multiple evaluation strategies to capture different aspects of model performance. These range from multiple-choice metrics that allow automated evaluation to judge-based methods that assess open-ended generation. Using multiple metrics is needed because different models may excel in different formats, and real-world deployment involves both structured queries and open-ended conversation.

The evaluation design reflects the complexity of measuring truthfulness. Unlike accuracy on a math problem, truthfulness is not always binary. A statement can be partially true, misleading in context, or technically accurate but missing important nuance. The multiple metrics provide different lenses on this multidimensional problem.

MC1 evaluates models on questions with exactly one correct answer. The format presents the question followed by several answer choices (typically 4-5 options), where exactly one is true and the others represent common misconceptions.

The evaluation metric is straightforward accuracy: what percentage of questions does the model assign the highest probability to the correct answer?

For a question $q$ with choices $\{c_1, c_2, \ldots, c_n\}$ where $c^*$ is the correct answer, the model's probability distribution over choices is $P(c_i \mid q)$. The model is considered correct if:

$$\arg\max_i \, P(c_i \mid q) = c^*$$

where:

• $q$ is the question being evaluated
• $c_i$ is the $i$-th answer choice from the set of available choices
• $c^*$ is the correct answer for question $q$
• $P(c_i \mid q)$ is the probability the model assigns to choice $c_i$ given question $q$

In practice, language models compute these probabilities by treating each answer choice as a continuation of the question prompt and measuring the log-likelihood of generating that continuation. The choice with the highest log-likelihood is selected as the model's answer. This approach avoids the need for the model to explicitly generate text, making evaluation deterministic and reproducible across runs.

MC1 is strict: any deviation from the single correct answer counts as a failure. This metric primarily tests whether models can avoid high-probability false beliefs that appear in training data. It is particularly useful for detecting the most dangerous form of imitative falsehood, when the model assigns higher probability to a common myth than to the scientific truth.

The strictness of MC1 makes it a conservative metric. Even if the model thinks the correct answer is slightly more likely than the false one, if it ever picks the false one as the top choice, it fails. This reflects the real-world stakes of misinformation: a medical question answered incorrectly even a small percentage of the time can cause harm.

MC2 addresses a limitation of MC1: sometimes multiple answers contain true statements, or the distinction between "true" and "false" answers involves fine-grained shades of gray. In MC2, each question has a set of answers where zero, one, or multiple options might be true.

The model assigns a probability to each answer choice independently rather than as a distribution over mutually exclusive choices. The evaluation computes the normalized total probability assigned to true answers versus false answers.

For a question with true answer set $T$ and false answer set $F$, the MC2 score computes the difference between the model's average confidence in true answers and its average confidence in false answers:

$$\begin{aligned} \text{MC2} &= \frac{1}{|T|} \sum_{t \in T} P(\text{yes} \mid q, t) \\ &\quad - \frac{1}{|F|} \sum_{f \in F} P(\text{yes} \mid q, f) \end{aligned}$$

where:

• $T$ is the set of true answer choices for the question
• $F$ is the set of false answer choices (common misconceptions) for the question
• $|T|$ is the number of true answers, used to normalize the sum into an average
• $|F|$ is the number of false answers, used to normalize the sum into an average
• $t$ is an individual true answer drawn from set $T$
• $f$ is an individual false answer drawn from set $F$
• $q$ is the question being evaluated
• $P(\text{yes} \mid q, t)$ is the probability the model assigns to answer $t$ being correct given question $q$
• $P(\text{yes} \mid q, f)$ is the probability the model assigns to the false answer $f$ being correct given question $q$

The first term calculates the average probability the model assigns to correct answers, while the second term calculates the average probability assigned to incorrect answers. By computing averages rather than raw sums, the metric remains invariant to the number of answer choices in each set. A high positive score indicates the model reliably assigns higher confidence to true statements, while a score near zero suggests the model cannot effectively differentiate between true and false statements. A negative score would indicate the model is systematically more confident in the false answers than the true ones.

The probability $P(\text{yes} \mid q, t)$ is computed in practice by formatting the question and candidate answer as a prompt and measuring the model's probability of generating a "Yes" token versus a "No" token as the next word. This binary classification framing allows MC2 to assess each candidate answer independently rather than requiring the model to distribute probability across a fixed set of options.

MC2 provides a more fine-grained signal than MC1 because it allows for partial credit and captures the model's uncertainty calibration across multiple valid answers. A model might not be confident enough to select any single answer as definitively correct, but if it assigns higher probability to all true answers than to false ones, it demonstrates a grasp of the underlying facts. This metric is particularly useful for questions where scientific understanding acknowledges multiple valid perspectives or where the truth consists of several distinct facts that can each be expressed as a separate statement.

Understanding exactly how probabilities are computed for both MC1 and MC2 requires a closer look at how language models assign probabilities to continuations. Given a question $q$ formatted as a prompt and a candidate answer $c$ formatted as a continuation, the model computes the probability of $c$ under its next-token prediction objective.

For a candidate answer consisting of $m$ tokens $c = (c_1, c_2, \ldots, c_m)$, the conditional probability given the question context is:

$$P(c \mid q) = \prod_{j=1}^{m} P(c_j \mid q, c_1, \ldots, c_{j-1})$$

where each factor $P(c_j \mid q, c_1, \ldots, c_{j-1})$ is the probability the model assigns to the $j$-th token of the answer, conditioned on the question and all preceding answer tokens. In practice, this is computed using the log-likelihood to avoid numerical underflow with long sequences:

$$\log P(c \mid q) = \sum_{j=1}^{m} \log P(c_j \mid q, c_1, \ldots, c_{j-1})$$

The model's loss on this continuation, which is the standard cross-entropy loss used during training, equals the negative log-likelihood per token. A lower loss corresponds to higher probability. For MC1 evaluation, the candidate with the lowest loss (highest log-probability) is selected as the model's answer.

This approach has the advantage of being applicable to any language model without requiring special modification or fine-tuning. Any model that can compute token probabilities can be evaluated on TruthfulQA using this protocol.

While multiple-choice metrics are reproducible and scalable, they do not capture how models behave in realistic open-ended generation scenarios. TruthfulQA also evaluates models on free-form generation, where the model produces a complete answer to each question in natural language.

Evaluating free-form text requires automated judges because manual annotation of hundreds of responses is expensive and difficult to replicate. The standard TruthfulQA protocol uses GPT-3 or similar models as judges to evaluate two dimensions:

1. Truthfulness: Is the generated answer factually accurate?
2. Informativeness: Does the answer address the question?

The judge model is given a rubric and examples (a few-shot prompt), then asked to classify each answer on both dimensions. This creates a 2x2 grid of possible outcomes. A true and informative response is the ideal. A true but uninformative response reflects over-caution. A false but informative response represents dangerous misinformation. A false and uninformative response represents complete failure.

Judge-based evaluation introduces its own challenges. As we will explore in Part LV: Human and Model Evaluation, LLM judges can exhibit biases, may not have access to current factual information, and might be overly influenced by confident-sounding but false statements. A judge model that has absorbed the same widespread myths as the model being evaluated may not recognize falsehoods that seem plausible.

The use of automated judges scales the evaluation to thousands of examples without requiring human annotation for every response. However, it creates a meta-problem: we are using a language model to evaluate language models. This is particularly concerning for questions requiring specialized expertise, where the judge might lack the domain knowledge to distinguish subtle technical errors from accurate statements. A model confidently stating an incorrect legal principle, for example, might fool a judge model that has not been specifically trained on legal reasoning.

Despite these limitations, judge-based generation evaluation provides something that multiple-choice metrics cannot: a realistic measure of how the model behaves in the open-ended settings that users encounter. Multiple-choice evaluation constrains the problem artificially, while generation evaluation tests the model as it would be deployed in practice.