Experts: Transparency, human oversight and impact assessments needed to curb AI harms in hiring, housing and pricing

Consumer advocates and technologists told New Mexico’s Science, Technology & Telecommunications Committee that state action is needed to force transparency and human review of automated decision systems used in hiring, housing, insurance and other high‑stakes areas.

“We think about how smart safeguards actually make for better innovation and are not in opposition to AI innovation,” Grace Geddy, a policy analyst at Consumer Reports, told the committee. Matt Scherer of the Center for Democracy & Technology added, “Transparency is what drives innovation.”

Why it matters: Presenters said many commercial AI systems are built on training data that often reflect social biases and incomplete measures of job or creditworthiness. Without clearer disclosure requirements, consumers and workers cannot know whether they have been subject to an automated decision or how it reached its result, limiting their ability to seek redress.

What presenters said: The panel distinguished generative AI (which creates text or images) from predictive AI and algorithmic decision systems (ADS) that influence consequential outcomes. Scherer said ADSs span visible tools such as video interview platforms and hidden systems such as resume screeners and ad-targeting algorithms. He warned that companies sometimes claim “human review” while relying on minimal or no oversight.

The presenters cited concrete examples. Grace Geddy described a Consumer Reports investigation into Kroger’s analytics in Oregon, where consumers obtained detailed inferred profiles—family size, education, likelihood of taking a cruise—that were sold or used to personalize discounts. Scherer recounted Cigna’s use of an algorithm that led, over two months, to 300,000 payment denials at an average processing time of about 1.2 seconds per case, evidence he said that the advertised human review was a “rubber stamp.” The panel also noted high‑profile recruiting tools that favored irrelevant signals—names or hobbies—and said Amazon scrapped an internally biased recruiting prototype after testing revealed systematic discrimination.

Policy recommendations: Presenters urged states to require: clear disclosure when ADSs are used; impact assessments to check for civil‑rights, consumer‑protection and labor law violations; detailed information on the data inputs and the role an AI plays in decisions; and enforceable remedies such as private rights of action so individuals can pursue relief when agencies lack resources.

Limitations and tradeoffs: The witnesses acknowledged uncertainty about the reach of future AI capabilities and warned lawmakers against drafting overly narrow definitions that companies could exploit as loopholes. They pointed to New York City’s Local Law 144 as a cautionary tale where narrow coverage and weak enforcement led to poor compliance.

Next steps: Committee members asked about hallucinations in generative AI, chatbots in behavioral health, and how to balance insurance‑market exceptions. Presenters recommended targeted state action to enhance transparency and human oversight and encouraged lawmakers to study sector‑specific applications as they draft legislation.

The committee did not vote on any bills during the hearing; members recessed for lunch and signaled interest in drafting or refining state approaches to ADSs in the coming session.