Get Full Government Meeting Transcripts, Videos, & Alerts Forever!
Get email alerts on the Pretrial Risk Assessment topic
No spam. Unsubscribe anytime.
Whatcom County work group hears plans to validate local use of PSA pretrial risk tool
Summary
Dr. Zachary Hamilton outlined a statistical validation project for the Public Safety Assessment (PSA), describing measures the research team will use to test accuracy, time windows and potential bias in Whatcom County data; the project is funded in part through an AOC grant.
Judge Evan Jones convened a meeting of the Whatcom County Pretrial Processes Work Group and invited Dr. Zachary Hamilton to present on a project to validate the Public Safety Assessment (PSA) that local pretrial services have been using.
Dr. Hamilton, a professor at the University of Nebraska Omaha, told the group he will test whether the PSA’s risk scores predict three outcomes tracked by jurisdictions: failures to appear (FTA), new criminal filings and new violent criminal filings. “The tool itself ... is designed to score so that higher scores indicate a higher likelihood of each one of those three types of outcomes,” Hamilton said.
Why it matters: jurisdictions use PSA scores to guide release conditions and pretrial supervision. If a risk-assessment tool misclassifies people for the local population, it can affect who is detained or released and under what conditions.
Hamilton described the key steps of the planned validation: calculate the area under the curve (AUC) statistic to measure overall predictive accuracy; check that PSA risk categories (low, moderate, high) show increasing rates of the tracked outcomes; use survival models to account for differences in how long individuals remain in the community while their cases are pending; and test for bias in predictive performance across demographic groups.
Hamilton explained that the AUC ranges from about 0.50, equivalent to chance, to 1.00 for perfect prediction, and said field conventions treat small numerical increments meaningfully: roughly 0.50–0.55 as negligible, about 0.56–0.64 as small, roughly 0.64–0.71 as moderate and values above about 0.71 as strong predictive performance. He also said that, because the PSA was developed on a predominantly Kentucky sample, performance can vary when the tool is applied to populations that differ by race, gender or other characteristics.
Hamilton noted the project team requests not only summary scores but the individual PSA item responses for each assessed person so they can identify whether particular items are unreliable or perform differently in Whatcom County. He said some PSA items predict FTAs, some predict new criminal filings and some predict both; the PSA produces separate scores that are then combined into an overall category by taking the higher of the category scores.
On timing and methods, Hamilton said the team will use survival-analysis techniques to account for variation in how long cases remain open after release. “We take that to the PSA and we identify: does an individual who’s low risk end up not committing those events for longer periods of time? Does an individual that’s high risk commit them quicker?” he said.
Alan (staff member) told the group the validation is connected to a grant administered through the Administrative Office of the Courts (AOC). “Dr. Hamilton’s project is one of the things we’re working on with some funds through the AOC grant,” Alan said. The group discussed sharing the PSA developer’s public documentation; Hamilton said the underlying algorithm and publications are publicly available and offered to share materials with the work group.
There were no motions or votes. The meeting closed after brief updates about an upcoming AOC grant wrap-up presentation and a scheduled discussion about the future structure of the work group.
