Get Full Government Meeting Transcripts, Videos, & Alerts Forever!
Get email alerts on the Qmm Scoring topic
No spam. Unsubscribe anytime.
Presenter explains scoring approach: three assessors, substantiation, and reconciliation to limit bias
Summary
The presenter outlined steps to reduce subjectivity in scoring: three independent assessors per establishment, a standard rubric of questions, documented substantiation for responses, and a reconciliation process to resolve variability before assigning final scores.
The agency official described the programs approach to scoring maturity to limit subjectivity and bias. Each establishment was independently scored by three assessors using a standard set of questions and a five-level rubric; responses had to be substantiated with supporting evidence for full credit.
The official said integrated scoring variability was monitored and flagged, and teams revisited assessment materials to resolve discrepancies before assigning final scores. "All assessors use a standard set of questions and a scoring rubric to evaluate and score the establishment's maturity," the presenter said, noting assessor collaboration during resolution of discrepancies.
The presenter also described that some assessment questions required simplification or rewording to reduce ambiguity and variability among assessors as the tool is refined for later cohorts.
