Intersectional Fairness Measurement in Hiring Algorithms

confidence: high · 9 sources

⚠
Working research — not a recommendation.

This page records what the evidence says a policy does, not what anyone should do about it. It has been synthesized from the cited sources but has not been independently reviewed.

What the evidence says

Single-axis audits can miss bias that a combined analysis shows. A survey in the sources states that separating gender groups by race exposes bias toward dark-skinned females that dark-skinned males would otherwise mask. A second survey states that intersectional and subgroup analyses are necessary to reveal compounded harms. Subgroup fairness applies a group constraint, such as equal false positive rates, across a large collection of subgroups. None of these claims gives a measured size for the bias that single-axis analysis misses.

Detection tools perform worse on intersections than on single attributes. In one comparison, classical statistical methods reach 72–78% accuracy for single-bias detection. For intersectional bias that falls to 45–58%. A multi-task adversarial framework reaches 83.7% for intersectional bias, 15.7 percentage points above the best baseline. The baselines score 61–68%. Removing its multi-task learning module lowers that accuracy from 83.7% to 69.4%. These are results from one study, and the claims flag that they are untested on non-Western data and on extreme class imbalance.

For hiring pipelines, theory and simulation both show that a fairness rule at one stage does not carry through to the final hire. An equal-gender shortlist constraint raises the share of women from an average of 31% in the applicant pool to 50% on the shortlist, by design. Among finalists the share is 41%, a modest gain. The mechanism is correlation between the algorithm's screening score and the hiring manager's score. Higher correlation lowers the female share of final hires. A regression on 254 job postings gives a coefficient of -0.7005 on that correlation. A gender gap in how well the algorithm predicts manager scores also lowers the female share, with a coefficient of -0.2896. A separate two-stage analysis finds that fair components may not combine into a fair pipeline. It also finds that a first stage that grants few interviews to a minority class limits what the second stage can fix.

Human-subject evidence shows that employer traits shape how far fair ranking helps. Fair ranking algorithms increase the number of underrepresented candidates selected. That gain is dampened in job contexts where employers hold a persistent gender preference, such as moving assistance jobs. In those jobs, the female × moving-job interaction is negative (p < .001). Selection of female candidates also varies across employer age, income and education intersections. Younger male employers earning below $60k with no academic degree select fewer female candidates. Female employers with the same profile select more. Female employers with the same age and income but an academic degree select fewer.

Several expected outcomes have thin evidence. No claim measures how intersectional fairness audits change real hiring outcomes. The claim on LLM resume screening is a taxonomy example without data. The claim on disability and job-matching algorithms rests on a cited study whose design the source does not describe.

Where the evidence splits

No contradictions in the current sources.

Thresholds and dose-response

The 80% rule is the one cutoff stated in the sources. Selection procedures sometimes use it to decide whether a process has disparate effects. A ratio smaller than 0.8 marks a process as having them. The source states this as a legal origin. It does not say whether the cutoff applies to intersectional subgroups.

Correlation between algorithmic and manager scores shows a dose-response direction. A higher correlation (θ) gives a lower female share of final hires. The sources give no cutoff value for it.

Open empirical questions

Findings

Disaggregating by several demographic axes at once reveals bias that single-axis analysis conceals

confidence — direction: highconfidence — magnitude: lowconsensus: settled in direction, thin on magnitude. Both sources lean the same way.
mechanism
A single-axis group can average a harmed subgroup with a favored one. Dark-skinned males would offset dark-skinned females in a gender-only or race-only comparison.
effect
Reveals bias not visible in single-axis analysis. Magnitude is not quantified.
conditions / moderators
Depends on the cited methodology in the survey. Intersectional identities are likely to be affected by multiple bias-conducive factors. Not otherwise established.

Subgroup fairness constraints apply a group fairness criterion across many subgroups at once

confidence — direction: highconfidence — magnitude: highconsensus: settled. The claims are definitional.
mechanism
The method picks a group constraint, such as equalizing false positives, and tests whether it holds over a large collection of subgroups. The (1+ε)-equal opportunity definition extends to several protected classes. Each class gets its own parameter relative to the majority class.
effect
No measured effect. These are definitions.
conditions / moderators
Whether the multiplicative extension holds when protected attributes interact or correlate is not established.

A multi-task adversarial framework increases intersectional bias detection accuracy over classical and baseline methods

confidence — direction: highconfidence — magnitude: mediumconsensus: thin. It is a single study.
mechanism
Multi-task learning with attention mechanisms models multi-attribute interactions. Classical methods do not capture the patterns that those interactions create.
effect
Increases accuracy by 12–18 percentage points over traditional methods. Reaches 83.7% for intersectional bias, 15.7 points above the best baseline (61–68%). Classical methods fall from 72–78% on single bias to 45–58% on intersectional bias. Removing the multi-task module decreases accuracy from 83.7% to 69.4%, a drop of 14.3 points.
conditions / moderators
Tested on three datasets with five-fold cross-validation. Unknown for non-Western occupational data and for class imbalance above 20:1. Unknown whether the ablation size holds on the other two datasets.

Equal-gender shortlist constraints produce a modest increase in the female share of final hires, which decreases as algorithm–manager score correlation increases

confidence — direction: highconfidence — magnitude: highconsensus: thin. It is a single study, though its theory and data agree.
mechanism
The shortlist is balanced by design. The hiring manager then selects among shortlisted candidates, and the manager's scores correlate with the screening algorithm's scores. A gender gap in that correlation (δ > 0) lowers the manager's scores for shortlisted women.
effect
Raises the female share from 31% in the applicant pool to 50% on the shortlist and 41% among finalists, a 10 percentage-point gain. Higher correlation (θ) decreases the female share of hires (β₁ = -0.7005, p<0.05, 254 postings). A larger gender gap in correlation (δ) decreases it (β₂ = -0.2896, p<0.01).
conditions / moderators
Empirical simulation on 799,000 applicants across 8 firms and 3,608 job postings. Equal-gender shortlists do not guarantee equal-gender hires even with no gender bias at the hiring stage.

Fair components may not compose into a fair multi-stage pipeline, and early-stage unfairness limits later correction

confidence — direction: highconfidence — magnitude: lowconsensus: thin. It is a single theoretical source.
mechanism
In a two-stage filter, the second stage can only select from those the first stage passes. Fairness parameters multiply across stages as (1+ε)(1+δ). A first stage that interviews only one or two minority candidates caps what the second stage can do. Qualification criteria also differ between stages, so a candidate can be hire-qualified but not interview-qualified.
effect
Composition fails to guarantee fairness under (1+ε)-equal opportunity. Downstream fairness measures cannot necessarily overcome upstream bias. Simulation finds the final outcome of a (1+ε)-decision is independent of the stage where it is applied.
conditions / moderators
Theoretical results for two stages. Not established for longer pipelines or other fairness definitions, or for real stages with varying capacity and qualification distributions.

Fair ranking increases selection of underrepresented candidates, but this is dampened where employers hold persistent gender preferences

confidence — direction: mediumconfidence — magnitude: lowconsensus: thin. It is a single experimental study, plus a survey summary on employer preferences.
mechanism
Employer preferences within a job context counteract the reranking. Female × moving-job interaction terms capture this. Employers also prefer culturally similar candidates.
effect
Increases the number of underrepresented candidates selected. Female candidates are less likely to be chosen for moving assistance than for event staffing jobs (interaction p < .001). Females benefit more from positive reviews than males (p < .001). Selection of female candidates varies by employer gender, age, income and education (p-values from .001 to .008 in the claim).
conditions / moderators
Depends on job context and on employer demographic intersection. Employer similarity preferences may increase discrimination where decision-makers are demographically homogeneous. That claim is hedged, and the source does not demonstrate it.

Optimizing for between-group fairness worsens within-group fairness, and common fairness criteria cannot be combined

confidence — direction: highconfidence — magnitude: lowconsensus: thin. One review summarizes cited work.
mechanism
Group fairness methods that address only between-group issues leave within-group issues unaddressed. Established definitions of different fairness ideals have been shown to be mathematically intractable in combination. Anonymization approaches can also struggle with several sensitive attributes at once.
effect
Decreases within-group fairness when between-group fairness is prioritized. Combining criteria is mathematically intractable. Magnitudes are not given.
conditions / moderators
The sources do not say whether the cited demonstration generalizes across fairness metrics and datasets.

Hiring algorithms are reported to produce unequal outcomes for disabled candidates and to preserve existing inequities

confidence — direction: mediumconfidence — magnitude: lowconsensus: thin
mechanism
Job-matching algorithms penalize non-standard educational timelines associated with disability. LLM-aided resume screening is said to preserve existing hiring inequities. The source gives no mechanism for the second claim.
effect
Penalization of disabled users, with no magnitude. Preservation of inequities, with no magnitude.
conditions / moderators
The disability claim is cited from another study whose design is not stated. The LLM claim is a taxonomy-table example without data.