Kansas test standard-setting panels recommend new cut-score impacts after weeklong review; board to adopt scale in August

TOPEKA '1 Kansas State Board of Education members on July 8 received the technical outcome of a June standard-setting meeting that recommended new cut scores and impact distributions for the redesigned state English-language arts, math and science assessments.

The standard-setting exercise used the widely accepted "bookmark" method. The process brought 141 Kansas educators together in grade-band panels to map test items (ordered by difficulty) to performance-level descriptions drafted by the department and to place threshold bookmarks for performance levels. Panels then reviewed impact data and participated in cross-grade articulation before producing recommended cut points.

KSDE contracted with the University of Kansas and an outside psychometrician to facilitate the work and to observe the technical process. An independent technical advisory member from the Human Resources Research Organization (HUMRO) observed the meetings and reported the process followed accepted standards for transparency and consistency.

After the workshop, KSDE's assessment team presented the recommended impacts to the board as distributions of students expected at each performance level under the proposed cut scores. Under the proposed ELA distribution, roughly one-quarter of students would be in Level 1, about roughly half in Level 2, and approximately one in ten in Level 4, with the remainder in Level 3 (proficient). Math and science impact distributions were presented grade-by-grade; staff and contractors noted differences in difficulty and student performance patterns at mid-grade transitions (for example between grades 6 and 7 in math).

KSDE officials emphasized that the board had not yet adopted a numeric scale. The board received the technical report and impact tables at the July meeting; KSDE staff said the formal numeric scale will be presented for adoption at the board's August meeting, followed by parent and district score reports when adopted.

Why it matters: Standard-setting determines how raw test scores translate into performance levels that are reported to parents, districts and the public. The process the department used '1 panel recruitment with geographic and role diversity, iterative rounds of independent bookmarking, the provision of NAEP/ACT and prior-year data, and a final vertical articulation panel '1 is consistent with accepted psychometric practice.

What's next: KSDE will present the numeric scale for board adoption at the August meeting and will then finalize score reports for the spring 2025 administration. Department staff and KU contractors said grade-by-grade performance and the grade 11 science test will continue to receive technical review; staff noted the grade 11 science administration produced item performance that made standard setting difficult and that they plan further item-review work on the science instrument.

Panelists and outside observers praised the quality of the participating Kansas educators, who reported that the workshop increased their assessment literacy and that the process for arriving at recommended cut scores was fair and well facilitated.

Key technical points - Bookmark method: Panels mapped items ordered by difficulty and placed bookmarks for threshold performance. - Vertical articulation: Panel representatives reviewed grade-to-grade consistency to avoid implausible jumps between adjacent grades. - Impact data: Panelists saw how their bookmarks translated to statewide distributions and discussed policy implications before final recommendations.

Speakers: Beth Fultz (Director, Career Standards & Assessments), Dr. Carla Egan (standard-setting leader, KU), Matt Copeland (KU Assessment & Technology Solutions), Dr. Art Thacker (HUMRO technical observer), Dr. Dan Graman (Kansas Assessment Advisory Council).