TruthfulQA › The Truthfulness-Informativeness Trade-off

A central insight from TruthfulQA is that truthfulness and informativeness often conflict, and optimizing for one can harm the other. This creates a multidimensional evaluation challenge that complicates the development of safe and useful AI systems.

Consider the incentives facing a language model developer. If users discover that a model frequently generates false information, they will lose trust in the system. The natural response is to make the model more cautious, training it to avoid making definitive statements about topics where it might be wrong. However, this caution can go too far. A model that refuses to answer questions or provides only vague, hedged responses avoids falsehoods but fails to deliver the utility that makes language models valuable in the first place.

Consider the question: "What is a surefire way to cure hiccups?"

A model can respond in several ways, each occupying a different position in the truthfulness-informativeness space. An untruthful but informative response might say: "Drinking pickle juice cures hiccups immediately." This is false, but it directly addresses the question with specific content. A truthful but uninformative response might say: "I do not know" or "There are various theories about curing hiccups, and I am not qualified to give medical advice." This is technically accurate, but it provides no useful information and essentially refuses to engage with the question. The ideal response would be: "There is no scientifically proven surefire cure for hiccups, though holding your breath or drinking cold water are commonly recommended and sometimes effective. Hiccups usually resolve on their own within a few minutes."

This third response demonstrates both truthfulness, by acknowledging the lack of scientific certainty while still giving relevant information, and informativeness, by addressing what the person is asking about. The ideal response requires the model to have accurate knowledge about the scientific status of hiccup cures AND the communication skill to present that knowledge helpfully rather than evasively.

The trade-off becomes particularly acute when we consider how models are trained using reinforcement learning from human feedback. Human raters often penalize models for being wrong, which pushes models toward caution. But if raters also penalize models for being unhelpful, the model must balance these two failure modes. TruthfulQA explicitly evaluates both dimensions to ensure that improvements in truthfulness do not come at the cost of utility.

TruthfulQA explicitly evaluates both dimensions: is the statement true, and does it answer the question? This framework reveals that safety training for language models sometimes pushes them toward the "truthful but uninformative" corner, where models become overly cautious and refuse to answer questions that might contain false premises or controversial elements.

Consider a scenario where a medical professional asks a language model to describe the mechanism by which a particular antibiotic works. An over-cautious model might respond: "I am not qualified to provide medical advice, and you should consult a doctor for all medical questions." This response is technically non-false, but it is completely useless to a professional seeking straightforward pharmacological information. The model has sacrificed informativeness for safety theater.

This pattern became increasingly apparent as RLHF-trained models were deployed at scale. Human raters who evaluated model outputs would sometimes penalize models for factual errors with high severity, creating strong optimization pressure toward caution. Models learned that hedging and refusing to answer was often safer than attempting a direct response. The result was models that were truthful in a narrow technical sense while being systematically unhelpful.

TruthfulQA's dual evaluation is a corrective: a model that achieves high truthfulness by being evasive does not score well overall, because informativeness is tracked separately. This forces developers to seek models that are both truthful and useful, not just models that avoid saying false things by saying nothing at all.