California lawmakers, clinicians and tech leaders weigh generative AI’s promise and risks for health care

A joint informational hearing of the California State Assembly’s Health Committee and Privacy Committee gathered experts, health system leaders, patient advocates and labor representatives to examine current uses of generative artificial intelligence in health care, potential benefits and documented risks, and the role state policy could play in ensuring equitable, safe deployment.

The hearing opened with Chair Bonta and Chair Bauer Kahan urging lawmakers to consider both opportunity and risk as AI tools — from ambient scribes to imaging triage and predictive risk models — are adopted. "We have a responsibility to pay attention to all of these developments, ask these questions, and help guide the technology in ways that maximize benefit to Californians and minimize harm," Chair Bonta said.

The witnesses described real-world deployments and measurable early impacts. Craig Kwiatkowski, speaking for Cedars‑Sinai, said many AI tools are designed to "extend, enhance, assist, and augment" clinicians and stressed organizational governance of AI at his health system. Kaiser Permanente’s Daniel Yang said his system has deployed an ambient scribe to more than 25,000 doctors and reported that the tool has been used in over 6 million encounters; he acknowledged testing and ongoing monitoring but added the deployment was intended to reduce clinician administrative burden and improve patient–clinician interaction. "There’s only one way for me to guarantee that we can eliminate risks associated with AI in health care," Yang said, "and that is for me to never deploy these AI technologies at all," a position he said would, however, accept the status quo of capacity limits.

Vendors and developers framed different near‑term priorities. Fawad Bhatt, founder of Penguin AI, urged starting with administrative and back‑office automation — citing his estimate that the administrative burden in U.S. health care "costs us $1,000,000,000,000 a year" — and treating clinical use cases with the same rigor as drug development. Amy McDonough of Google Health emphasized product portfolios and partnerships aimed at improving information access and clinician workflows, while noting a focus on responsible model development.

Researchers and ethicists warned that poor design and limited data can lock in harms and disparities. Ziad Obermeyer (University of California, Berkeley) recounted finding racial bias in a widely used risk algorithm that had predicted cost rather than sickness, excluding many patients in need from outreach programs. He argued for greater data access for researchers and for public programs, such as Medi‑Cal, to use purchasing power to steer development toward high‑value, equitable applications. "Every year, 35,000 people in California just drop dead," Obermeyer said, arguing that better predictive tools could save lives if developed and deployed with care.

Speakers described clinical wins and limits. Cedars‑Sinai presented imaging workflows that triage scans and text clinicians when urgent action is indicated, reporting a roughly 40% reduction in time to mechanical thrombectomy in one workflow. Cedars also described a maternal‑fetal decision‑support tool that, in preliminary work, predicted delivery mode with about 90% accuracy in the first four hours, but clinicians on the panel insisted such tools should supplement — not replace — clinician judgment and be measured for effects on care decisions and disparities.

Safety‑net concerns and equity were recurring themes. Cara Carter of the California Health Care Foundation and representatives from community clinic networks stressed that Medi‑Cal and safety‑net providers frequently lack the capital, data infrastructure and bargaining leverage of large systems, making them vulnerable to becoming AI "have nots." CHCF cited that roughly 15,000,000 Californians rely on Medi‑Cal or remain uninsured and urged state support for data infrastructure, workforce training, governance and funding models to prevent widening disparities.

Labor and patient‑rights groups pressed for worker protections and patient safeguards. The California Nurses Association cautioned that AI could be used to justify understaffing or to deskill important clinical judgment, and the Light Collective urged enforceable patient rights including transparency about AI use, patient participation in governance, and legal recourse for harms.

On regulation and accountability, Stanford’s Michelle Mello recommended a governance requirement tied to licensure—analogous to institutional review boards for research—so that hospitals must have formal review and monitoring processes for AI tools. Multiple panelists pointed to existing federal tools (for example, HIPAA and civil‑rights laws) but emphasized a state role in clarifying liability, mandating disclosure standards (such as model cards), and funding readiness assistance for lower‑resourced providers. Several witnesses cautioned that unanswered federal proposals (including a debated moratorium on state AI regulation under HR 1) could affect states’ options, and some urged California to act now to set standards for transparency, bias testing, monitoring, and equitable procurement.

Where the state could act, witnesses suggested: incentivizing inclusive data sharing and privacy‑preserving collaborations, requiring or encouraging standardized disclosures about model purpose and performance, funding technical assistance and secure cloud/EHR integrations for safety‑net providers, and using Medi‑Cal purchasing power to set outcome and equity targets for clinical AI. Several panelists also urged requiring human review and explicit monitoring metrics so that "human‑in‑the‑loop" checks do not atrophy into blind reliance.

The hearing did not produce formal votes or directives; chairs asked panelists to share follow‑up materials. Public commenters reiterated fears about data misuse and urged a balanced approach that protects privacy while enabling life‑saving research.

As the panelists and members agreed, generative AI is already affecting care delivery in California. The debate at the Assembly centered less on whether to use the technology and more on how to manage adoption so it improves outcomes without amplifying inequities, eroding patient privacy, or undermining clinical judgment. Several witnesses offered concrete near‑term policy options: standardized vendor disclosures, state‑supported infrastructure and training for Medi‑Cal providers, and licensing‑linked governance requirements for health facilities. Lawmakers said they will review the materials and return to policy options that balance innovation with protections for patients and the workforce.