State auditor urges Vermont to tighten how the Vermont Training Program reports outcomes
Get AI-powered insights, summaries, and transcripts
Sign Up Free
Summary
State Auditor Doug Hoffer told the House Commerce & Economic Development Committee that the Vermont Training Programs current performance measures—reporting trainee income changes using medians over five quarters—can mislead; he recommended comparing like occupations, reporting distributional detail, and adding caveats about alternative explanations for wage gains.
State Auditor Doug Hoffer told the Vermont House Committee on Commerce & Economic Development on Jan. 27 that the Vermont Training Programs (VTP) performance measures as currently reported risk overstating program impact.
Hoffer said the program reports changes in trainees quarterly income over a five-quarter window and presents median income increases without distributional detail, rather than reporting hourly wages. "You're really not getting wage data, you're getting income data," Hoffer said, noting the reports assume a 40-hour workweek and therefore can be distorted by changes in hours worked.
Hoffer also warned the committee that comparing trainees outcomes to statewide averages is not an "apples-to-apples" comparison because trainees are concentrated in particular industries. He said 67% of trainees in the most recent year were in what the program labeled "advanced manufacturing," while manufacturing accounts for roughly 11% of statewide employment, a mismatch he said can skew apparent program gains.
Hoffer questioned the unusual choice to compare outcomes over five quarters rather than the more common four-quarter adjustment labor-market analysts use, saying seasonal and quarterly differences can distort comparisons when not adjusted properly. He also said the VTP reports increase figures without caveats, which could lead lay readers to conclude the program alone caused income changes even though other explanations exist: many trainees are new hires, probationary raises can raise pay within the measurement window, and changes in hours worked can alter income independent of wage rates.
Hoffer said the programs annual budget is modest in size (transcript: "about 1.2 a year"; units not specified in the record) but cautioned that repeated small awards to already-successful firms could create entrenched interests. He named past recipients including Green Mountain Coffee (now Keurig), Dealer dot com, and "beta," which he said later became a large company.
Committee members asked Hoffer for materials and standards to evaluate workforce programs. Hoffer said his office would share a prior review of Department of Labor programs and other literature on workforce education and training evaluation and encouraged committee members to press program officials about methodology when they present results.
The hearing concluded with members thanking Hoffer and requesting the follow-up materials he offered. Hoffer said he will provide the committee and staff with additional memos and background work before he leaves office.