Field-Deployed Algorithmic Pretrial Risk-Assessment Outcomes

What the evidence says

Across three widely-studied instruments — COMPAS, the Public Safety Assessment (PSA), and the Virginia Pretrial Risk Assessment Instrument (VPRAI) — the evidence agrees on direction more than it disagrees: field-deployed pretrial risk tools show measurable racial disparities in error rates, and those disparities carry through into judicial decisions rather than being neutralized by them. On COMPAS, Black defendants are almost twice as likely as white defendants to be labeled high-risk despite not reoffending, and twice as likely to be misclassified as high-risk for violent recidivism specifically, while white defendants misclassified as low-risk go on to reoffend at a rate one source reports as 63% higher than the equivalent Black-defendant misclassification rate. On PSA, the failure-to-appear scale is less predictive for Black defendants (ROC = 0.612) than for white defendants (ROC = 0.655), even though the tool's calibration is not measurably different by race for two of its three risk scores. VPRAI shows the sharpest documented racial gap in recommendations: among defendants who are in fact at risk of reoffending or failing to appear, VPRAI recommends detention for 86.0% of Black defendants versus 49.3% of white defendants, a 36.7-percentage-point gap.

On raw predictive accuracy, the evidence is more settled and less contested: interpretable models using only age, gender, and prior record show no measured accuracy difference from COMPAS, which can draw on up to 137 inputs, and PSA's overall predictive utility (about 64%) matches COMPAS. Model complexity, in other words, is not what is producing the disparities described above — the disparities show up in tools regardless of how many inputs they use.

Judicial behavior compounds rather than corrects these gaps. Judges follow PSA "do not release" recommendations in over 90% of cases, and 80% of surveyed judges report that PSA "always" or "often" informs their release decision — high, consistent reliance on the tool's output. But that reliance is asymmetric: with VPRAI, judges correctly detain in 84.9% of cases carrying a detention recommendation, but correctly release in only 54.7% of cases carrying a release recommendation, and this asymmetry falls unevenly by race. Among defendants not actually at risk who received a VPRAI release recommendation, white defendants are released 60.3% of the time versus 49.0% of Black defendants, and wrongful detention (detaining someone with no true pretrial-failure risk) occurs in 51.4% of Black-defendant cases versus 39.7% of white-defendant cases.

At the aggregate level, the evidence on what these tools actually do to incarceration is thin and points against the expectation that motivated their adoption: recent findings show either no measured effect on incarceration rates at all, or a decrease much smaller than initially hoped. The literature attributes this in large part to heterogeneous judicial compliance with tool recommendations, and one study ties differential compliance — judges in whiter counties complying more, and overrides of low/moderate-risk scores happening more often for Black than white defendants — directly to an increased racial gap in incarceration after deployment. On PSA specifically, deployment in San Francisco is reported to coincide with increased release rates at pre-arraignment and early-stage proceedings, though this rests on a single interview rather than a quantitative before/after comparison.

One outcome the sources do not cover with any specificity: the magnitude of PSA's gender disparity. The evidence establishes that PSA's new-violent-criminal-activity score produces a higher false-positive rate for women than men, but no claim here quantifies that gap the way the racial disparities are quantified.

Where the evidence splits

Findings on the aggregate incarceration effect of risk-tool deployment disagree on magnitude: one line of research finds no impact at all on incarceration rates, while another finds a decrease, but of much smaller magnitude than initially expected. The source citing both does not reconcile which is correct or what separates the two results — it offers heterogeneous judicial compliance as a general explanation for why outcomes fall short of expectations, but does not explain why some studies land on zero effect and others on a small-but-nonzero one. That specific gap is unexplained by the sources on this page.

PSA's own results show a tension between calibration and discrimination that the source itself explains: PSA is reported as relatively well calibrated between white and Black defendants for two of its three risk scores, yet its failure-to-appear scale is measurably less predictive for Black defendants (ROC 0.612) than white defendants (ROC 0.655). The source notes that calibration parity and error-rate balance are mathematically incompatible — a tool can satisfy one criterion or the other but not necessarily both at once — which is why PSA can show no measured calibration gap while simultaneously showing a discrimination gap.

Thresholds and dose-response

At the highest PSA risk tier (score 5, the "do not release"/highest-risk classification), the false-positive rate rises sharply and varies by outcome type: defendants classified high-risk for failure to appear have a 75% false positive rate (they do not, in fact, fail to appear), and defendants classified high-risk for new violent criminal activity have a 97% false positive rate. The claims note this is driven in part by the low base rates of the underlying behavior (failure-to-appear and violent recidivism are both rare), which mechanically inflates false-positive rates at the top of the risk scale.

Open empirical questions

• Whether PSA's failure-to-appear discrimination gap (ROC 0.612 vs. 0.655) and VPRAI's racial detention-recommendation gap generalize beyond the jurisdictions studied here.
• What magnitude underlies PSA's reported gender disparity in new-violent-criminal-activity scoring — the claims establish direction but not size.
• Whether the racial error-rate disparities in COMPAS and PSA are driven by differing base rates of actual recidivism/failure-to-appear between racial groups, or persist after adjusting for base rates.
• Why some studies of risk-tool deployment find no measured effect on incarceration while others find a smaller-than-expected decrease — the sources here do not identify what separates these results.