Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Fairness Metrics in the Context of Language Models

The fairness metrics discussed so far have been framed around binary classification, which covers many high-stakes applications like hiring, lending, and medical diagnosis. But language models raise additional fairness challenges that these metrics do not directly address.

When a language model generates text, it does not produce binary predictions. It assigns probabilities to tokens, sentences, and continuations. Measuring fairness in generative settings requires adapting the binary classification framework or developing new approaches. Several directions have emerged:

Counterfactual input perturbation. One approach is to measure how model outputs change when a sensitive attribute in the input is swapped. Does a language model give different career advice when the user's described gender changes? Do summarization systems preserve positive framing about one group and negative framing about another? These tests operationalize a form of counterfactual fairness adapted to text generation.

Stereotype and association probing. Another approach measures whether the model associates certain demographic groups with certain roles, occupations, or characteristics at different rates. Word association tasks and template-filling probes like "The [doctor/nurse] was [adjective]" measure these associations directly. These tests are related to but distinct from classification fairness metrics; they measure representation harms rather than classification outcome disparities.

Toxicity and sentiment disparities. Metrics like the probability of generating toxic continuations, or sentiment differences in auto-completed text, vary across demographic groups mentioned in the prompt. These disparities can be quantified with group-level comparisons analogous to demographic parity, adapted to probability distributions over text rather than binary outcomes.

Embedding-level fairness. Representation bias in word and sentence embeddings can be measured using geometry. The WEAT (Word Embedding Association Test) and related methods measure the cosine similarity between demographic group terms and attribute terms, testing whether the embedding space encodes stereotype associations. These are related to but different from outcome-based fairness metrics; they measure what the model has learned to represent, rather than what decisions it makes.

These generative and representation-based fairness concerns are covered in more detail in the chapter on Representation Harms. The key point is that the fairness metric framework developed here, grounded in classification outcomes and group statistics, is a necessary starting point but must be extended for the full range of language model applications.