Get Full Government Meeting Transcripts, Videos, & Alerts Forever!
Get email alerts on the Ai Evaluation topic
No spam. Unsubscribe anytime.
Fed convenes industry and nonprofits on AI: designers call for 'evals' and human‑in‑the‑loop safeguards
Summary
A Fed panel led by Governor Lisa Cook explored how AI can expand financial inclusion if products are designed around user trust, evaluated with applied 'evals' and deployed with human oversight; SaverLife reported strong early uptake of its AI navigator.
A panel on artificial intelligence and financial inclusion at the Federal Reserve featured private‑sector, nonprofit and academic practitioners who urged a design‑first, empirically tested approach to AI products for low‑ and moderate‑income consumers.
Governor Lisa Cook (Board of Governors) moderated the session and said the Fed cares about inclusive access to new financial tools. Panelists described how behavioral science and careful product design can make AI tools useful and trustworthy for people with scarce time, attention and trust in institutions.
Laura Blattner of MIT’s Bike Shop applied AI lab argued that many AI products fail because they focus on technology rather than the behavioral barriers people face. She urged teams to build applied evaluation frameworks — “evals” — that test whether a tool can gather the right information, ask efficient questions and make a single, actionable recommendation to a user.
Sarah of SaverLife described the nonprofit’s AI navigator pilot. She said members who received AI‑generated, personalized recommendations were far more likely to act on them: "for those that received an AI generated recommendation, they're 10 times more likely to take up that recommendation," she reported, and described higher linkage rates and improved savings outcomes in early tests.
Sanjay Subramanian (PwC) and other industry speakers emphasized transparency, synthetic data practices when appropriate, and the need to monitor model drift. Panelists recommended iterative small pilots with human oversight, and said guardrails — particularly around bias, privacy and credit decisions — are essential before scaling digital tools that influence financial outcomes.
The session combined concrete vendor experience, nonprofit outcomes and academic methods — and stressed that effectiveness depends on provider trustworthiness, user experience and the ability to evaluate products on real, difficult use cases rather than generic benchmarks.
Next steps suggested by panelists included shared evaluation standards, more cross‑sector partnerships to test real user flows, and regulatory engagement to set baseline expectations for higher‑risk AI applications.
