Consultants recommend four assessments and an evaluation checklist for Montana choice schools; commission to review report in April

Consultants from Solomon Research and Analytics presented recommended assessment options and an evaluation framework to the Board of Public Education during a virtual meeting, urging the commission to use a practical checklist and decision matrix when vetting assessment proposals from choice (charter) schools. "We have 4 recommended assessments that are approved for use," consultant Joe Marr told commissioners, outlining the firm's approach to meeting statutory requirements for validity and reliability.

Marr said the four recommended instruments are NWEA MAP Growth, the I‑Ready diagnostic (rebranded in coming years to I‑Ready Inform, the consultants said), the new state assessment referred to in the meeting as MAST, and ACT/preACT for high‑school measures. He described the technical basis for the selection—validity, reliability, subgroup comparability and growth models—and said the team supplied completed practical checklists and a decision matrix to help the commission evaluate proposed or alternate assessments.

Why this matters: the statute requires choice schools to include rigorous, valid and reliable indicators in contract performance frameworks. Marr told the commission that those statutory performance components include student academic proficiency, student academic growth, subgroup achievement gaps and postsecondary readiness; the consultants’ tools are intended to help the commission judge whether a school’s proposed assessments meet those statutory expectations.

The consultants laid out strengths and limitations for each recommended option. They described NWEA MAP Growth as a strong national norm‑referenced product with a popular conditional growth model and extensive technical documentation (the presenter cited roughly 15,000,000 student observations used in NWEA’s norming studies). I‑Ready was described as a criterion‑referenced diagnostic with detailed skill reporting and strong instructional alignment tools but with less publicly available technical documentation for some high‑school growth uses. The new state assessment (referred to in the meeting as MAST) was noted for being aligned to Montana standards and for including equity analyses that explicitly consider Native American students, but the presenters said it is early in statewide rollout and lacks an established growth model and independent alignment studies. The ACT/preACT pathway was characterized as an established college‑readiness measure that can support growth analyses only if administered in a laddered way (preACT then ACT), which may require additional test administrations beyond current state requirements.

Commissioners and members raised implementation concerns. Barbara Forrester Frank, speaking from classroom experience, said the state assessment has limited usefulness for day‑to‑day instruction: "We do not find it useful. We do not find it to drive instruction ... The reporting is terrible, and the kids don't take it seriously," she said. Consultant Joe Marr and others acknowledged such field critiques and advised the commission to treat state assessments like MAST as one possible data source while relying on multiple reference points and vendor documentation when making high‑stakes decisions.

The consultants emphasized next steps that the commission should consider before formal adoption of any tools: how and when raw data will be accessible for verification, whether the commission will require data‑sharing agreements or MOUs with schools or authorizers, the practical timing challenge of a June 30 reporting deadline for renewal years (spring assessment windows can extend into June), and whether the commission will adopt a staged reporting approach when data arrive late. Marr recommended establishing verification and administration protocols, participation‑rate expectations, and periodic audits or validation checks for school‑produced reports.

The Board of Public Education did not vote on or adopt any recommendations at the meeting. The chair said members should review the Solomon report and that the consultants’ recommendations will be on the April agenda for further discussion and possible action. McCall Flynn, the board’s executive director, reminded the group that the state board sets statewide assessments and that the ACT has long been used for high school; decisions about elementary measurements remain unresolved.

The consultants left the meeting noting they would provide copies of slides and the completed checklists and matrix. Commissioners asked for legal clarification on certain statutory language and for staff to consider data‑access arrangements as they move toward policy decisions.

Next steps: the commission will review the consultants’ report, seek legal guidance where needed and return to the topic at its April meeting; no final policy changes or votes were taken at this session.