NIST Statistician Urges Greater Transparency in Expert‑Witness Methods

Steve Lund, a statistician in the National Institute of Standards and Technology(NIST) Statistical Engineering Division, urged forensic practitioners to make the basis of expert-witness conclusions more transparent during a plenary presentation on statistical analysis and evidence interpretation.

Lund opened by reminding the audience that his remarks represented his personal views, "and do not represent any official policies or perspectives of NIST," and said the overarching goal for expert testimony should be to minimize wrongful convictions and false acquittals. He told the room that because there is no direct empirical path to measure that outcome, practitioners must emphasize secondary safeguards: clear methods, demonstrable performance and explicit acknowledgement of limits.

Why it matters: Lund argued that courts and investigators need more than assertions of "training and experience." He recommended compiling collections of reference instances where a method has been applied to cases with known ground truth and using straightforward visualizations—distribution plots and receiver-operating-characteristic (ROC) curves—to show how well a method discriminates same-source from different-source comparisons. "If you could reach that standard," he said, "a practitioner has transferred the basis of their expertise." He also quoted W. Edwards Deming: "what makes a scientist great is the care that they take in telling you what is wrong with their results so that you will not misuse them."

Details and evidence: Lund described practical components of transparent testimony: specify exactly what was done and what was observed; explain why a method was chosen; show example comparisons that produced particular scores; and, where methods overlap, explain alternatives and the strength of supporting data. He emphasized that statisticians often have a set of plausible models, not a single uniquely correct model, and that good practice includes characterizing how different reasonable models affect interpretation.

Caveats and tradeoffs: Lund noted that today's best method might be superseded by future methods or data and that precise interpretation requires both the right data and appropriate models. He framed a key tradeoff for experts and courts: whether testimony should emphasize the process and comparative performance or aim to give a single interpretation of meaning, and he warned that neither approach guarantees minimization of judicial error without transparent supporting evidence.

Session context: The remarks were part of a plenary session that included five presentations and a planned podcast-recorded Q&A. John Morgan described the session format and encouraged listeners to engage via the "Just Science" podcast.

What comes next: Lund concluded by urging presenters to "put their cards on the table" so informed audiences can judge the relevance and limits of the evidence. The session moved to the next presentation and a recorded podcast Q&A later in the program.