Panel and attendees press DOE on model hallucination and verification; formal validation emphasized

Multiple attendees raised concerns about model confidence and the risk of AI 'hallucinations' producing convincing but incorrect scientific outputs. Jan Vanauer (Stony Brook University) and others argued models should communicate uncertainty and confidence metrics to guide validation. "Maybe the platform can encourage that models actually give you a confidence score," Vanauer said.

Panelists described verification as a core design goal. Brian Spears and Rick Stevens noted the platform will combine formal verification, high-performance computing and experimental follow-up to test AI-generated hypotheses. "DOE is the formal verification piece for scientific hypothesis now," one panelist said, urging a closed loop: ideas generated by AI can be tested using HPC and experimental facilities to confirm or disprove results. The panel also suggested using model competitions, adversarial testing and ensemble approaches to drive skepticism and rigor into models' outputs.