Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Limitations and Practical Impact

Fairness metrics are necessary but not sufficient for building fair AI systems. Understanding their limitations prevents the false comfort of "my model passes the fairness audit."

Goodhart's Law applies to fairness. When demographic parity becomes a target metric, optimizing it can achieve the metric without achieving the underlying goal. A model that randomly upsamples predictions for a disadvantaged group can satisfy demographic parity without being fair in any individual sense. The phenomenon is well-documented: once a measure becomes a target, it ceases to be a good measure. In high-stakes settings, fairness audits should include qualitative review and stakeholder input alongside quantitative metric checks.

Metrics aggregate over individuals. Equalized odds says "on average, TPR is equal across groups." But individual members of each group may experience very different outcomes. A fair metric can mask extreme unfairness for specific subpopulations, especially when those subgroups are small or intersectional. A model that is perfectly fair for men and for white individuals overall can still systematically disadvantage white women or men of color in ways that aggregate statistics will not reveal unless you specifically look for them.

Ground truth is often itself biased. In criminal justice, healthcare, and hiring, the labels $Y$ used for training are produced by human systems with documented biases. Fairness metrics that condition on $Y$, including equalized odds and equal opportunity, will propagate those biases if $Y$ is not itself fair. Choosing metrics that are robust to label bias, or investing in better ground truth measurement, is often more impactful than optimizing existing metrics on biased labels.

Fairness metrics do not capture harm. The harm caused by a false positive in a parole decision, keeping someone incarcerated who would not reoffend, is qualitatively different from the harm of a false positive in a product recommendation, showing an ad someone does not want. Fairness metrics are dimensionless numbers; they do not encode the stakes of the decisions they evaluate. A fairness analysis is only as meaningful as the human judgment about what the errors cost for the people affected.

Sample size and statistical power matter. Fairness metrics estimated on small samples are noisy. A demographic parity difference of 0.05 computed on 50 individuals from each group carries wide confidence intervals and should not be treated as precise. In practice, fairness audits should report confidence intervals or conduct hypothesis tests, not just point estimates. When subgroups are small, as in intersectional analysis, the statistical power to detect real disparities may be low, which means failing to find a disparity is not the same as confirming fairness.

Despite these limitations, fairness metrics have had real impact. They created a common language for discussing algorithmic discrimination, enabling audits of deployed systems that previously operated without accountability. They influenced policy frameworks in the United States, European Union, and beyond, creating legal teeth for fairness obligations that previously had no measurement standard. They demonstrated to the industry that accuracy and fairness are not the same objective, a realization that prompted widespread adoption of fairness-aware modeling practices. The field has moved from intuitive notions of "this feels unfair" to quantitative, falsifiable claims, which is a prerequisite for systematic improvement even when the measurements are imperfect.