Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Choosing Fairness Metrics

Knowing that fairness metrics conflict, you have to make a deliberate choice. The impossibility result does not tell you which metric to prioritize. That is a values question that depends on your specific context, who bears the costs of different errors, and what legal and ethical obligations apply. The following considerations structure that decision.

False positives and false negatives carry different costs in different contexts, and in different directions for the individuals affected versus the institutions making decisions.

In a hiring context:

• A false positive means hiring someone who does not perform well. Cost to the institution: wasted resources, opportunity cost, possibly a legal claim if the hire is let go.
• A false negative means rejecting someone who would have performed well. Cost to the individual: a lost opportunity, potential discrimination claim.

If false negatives cause more harm (you are denying opportunity to qualified people), equal opportunity (equalizing TPR) is the more relevant criterion. If false positives cause more harm (you are providing a service or resource to people unlikely to benefit, at cost to those who would), equalizing FPR or achieving predictive parity matters more.

In criminal justice, the asymmetry runs in the other direction. A false positive in a recidivism risk tool means detaining or monitoring someone who would not have reoffended. The cost falls entirely on the individual being incorrectly classified as high-risk. A false negative means under-supervising someone who does reoffend. The costs there fall on the community and on the victim. Neither direction is obviously worse, which is why criminal justice applications generate the most contentious fairness debates.

Equalized odds requires both TPR and FPR to be equal, which protects both types of positive-class individuals (through equal TPR) and both types of negative-class individuals (through equal FPR). Predictive parity protects the institution making decisions: it ensures that positive predictions are equally reliable as guides to action, so the institution doesn't bear higher error costs when acting on predictions for one group versus another.

When the costs of errors fall primarily on the individuals being classified, as in parole decisions, medical diagnosis, or college admissions, individual-centric criteria like equalized odds or equal opportunity are more ethically appropriate. When the institution bears primary costs, for example a bank losing money on defaulted loans, predictive parity arguments carry more weight.

A useful test is to ask: if you were in the group with lower predicted probability, which fairness criterion would you want? Most people answer equalized odds or equal opportunity, because they want their actual qualifications to be recognized at the same rate as everyone else's. This perspective helps clarify the distributional ethics of the choice.

The impossibility theorem only bites when base rates differ between groups. But before accepting observed base rate differences as ground truth, it is worth asking why they differ.

In many real-world settings, observed base rate differences are themselves artifacts of biased data collection. Criminal recidivism data is shaped by policing patterns: if one community is over-policed, its members are more likely to have prior arrest records, not necessarily because they commit more crimes. Using such data as ground truth and then demanding predictive parity creates a circular argument: it encodes bias in the labels, calls the resulting model "calibrated," and declares the model fair.

Medical diagnosis datasets often have lower true positive rates for groups that were historically excluded from clinical trials or have lower access to healthcare. When such groups receive later diagnoses, it is not because they have lower rates of disease; it is because the diagnostic pathway is harder for them to complete. Training on such data and conditioning on $Y$ as if it were unbiased will propagate the healthcare access inequity.

If you suspect the ground truth labels $Y$ are themselves biased, group fairness metrics that condition on $Y$ (like equalized odds) can propagate rather than correct that bias. In these cases, demographic parity (which ignores $Y$) or causal approaches that try to reconstruct an unbiased outcome may be more appropriate, even though they are harder to justify mathematically in isolation.

In the United States, the Equal Credit Opportunity Act, Fair Housing Act, and Equal Employment Opportunity Act create legal fairness obligations that roughly align with demographic parity through the "four-fifths rule" for adverse impact. If the positive selection rate for any group is less than 80% of the rate for the highest-selected group, that constitutes prima facie evidence of adverse impact. The European Union's AI Act creates additional requirements for high-risk AI systems, requiring fundamental rights impact assessments and ongoing monitoring for bias.

These legal standards do not map cleanly onto any single mathematical definition of fairness. Legal compliance is a floor, not a ceiling. You may satisfy the four-fifths rule while still having large TPR disparities that cause real harm to individuals from under-selected groups. The law has not caught up with the full richness of the fairness metric literature, which means practitioners bear responsibility for thinking beyond legal compliance.

Most fairness definitions consider a single sensitive attribute at a time: race, or gender, or age, without their combination. This creates a significant blind spot. A person's experience of discrimination is shaped by individual identities and by the intersection of multiple identities simultaneously. Legal scholar Kimberlé Crenshaw coined the term "intersectionality" in 1989 to describe how Black women faced discrimination that was neither fully captured by analyses of race alone nor gender alone. Their specific combination of identities created unique vulnerabilities that single-axis analyses missed entirely.

The same problem arises in algorithmic fairness. A model may be fair for women overall and fair for Black individuals overall, but systematically unfair specifically for Black women as a subgroup. If the subgroup is small enough that it does not move aggregate statistics significantly, a standard group fairness audit will not catch the disparity. Worse, mitigation interventions designed to fix the racial disparity or the gender disparity independently may leave the intersection unchanged or even worsen it.

Intersectional fairness requires disaggregating metrics by combinations of sensitive attributes. Instead of checking $P(\hat{Y}=1 \mid A_{\text{race}}=\text{Black})$ and $P(\hat{Y}=1 \mid A_{\text{gender}}=\text{female})$ separately, you check $P(\hat{Y}=1 \mid A_{\text{race}}=\text{Black}, A_{\text{gender}}=\text{female})$ against all other combinations. This approach is significantly more data-intensive because intersectional subgroups can be small, leading to high variance in metric estimates. With $k$ binary sensitive attributes, there are $2^k$ subgroups to check, which creates both a statistical power problem and a multiple comparisons problem.

Several formal frameworks address intersectional fairness, including minimax fairness (optimizing for the worst-off subgroup rather than average group parity) and multi-calibration (requiring calibration within all subgroups simultaneously, including intersectional ones). These approaches acknowledge that "fair on average" can coexist with systematic unfairness for specific populations, and they shift the burden of proof toward the model developer to demonstrate fairness across all relevant subgroup combinations.

When selecting a fairness criterion for a specific application, work through these questions systematically:

• What are the positive and negative outcomes? Define clearly what $\hat{Y}=1$ means and who is helped or harmed by each type of error.
• What does ground truth $Y$ represent? Is it a reliable, unbiased measurement, or is it itself a product of historical bias?
• Who bears the cost of errors? The individuals being classified, or the institution making decisions?
• What are the legal requirements? What adverse impact rules or fairness mandates apply?
• What is the base rate situation? If base rates differ significantly across groups, which impossibility trade-offs are you willing to accept?
• Are there intersectional subgroups at risk? Are there small demographic intersections that might be systematically disadvantaged without appearing in aggregate statistics?

No framework answers these questions automatically. The choice of fairness metric is ultimately a values question wrapped in mathematics.