FDA study: many claims‑data real‑world analyses reach similar conclusions to randomized trials, with limits
Get AI-powered insights, summaries, and transcripts
Subscribe
Summary
An FDA‑funded demonstration that emulated randomized trials in U.S. claims data found many real‑world data (RWD) studies produced results similar to the original randomized controlled trials (RCTs)—a Pearson correlation of 0.82—while underscoring important limits related to data capture and emulation choices.
Dr. Marie Bradley, senior advisor for real‑world evidence at the FDA’s Office of Medical Policy, described results from an FDA‑funded demonstration project showing that many claims‑based real‑world data studies can reach conclusions similar to randomized controlled trials.
The project, called RCT‑duplicate, emulated 30 completed trials (23 pivotal and seven negative trials) using three U.S. claims databases — Optum Clinformatics Data Mart, IBM MarketScan and Medicare parts A, B and D. Analyses were run in each database and then combined. Bradley said the teams preregistered all emulation study protocols on ClinicalTrials.gov to reduce bias.
Why it matters: regulators and sponsors are assessing when RWD can inform regulatory decisions such as new indications or post‑approval studies. Bradley said the work “demonstrates that, in many cases, claims‑based RWD studies generally reach similar conclusions to RCTs,” and reported a Pearson correlation coefficient of 0.82 across the emulated pairs.
Key findings and limits
The team found that roughly half of the selected RCTs could be closely emulated in the claims data; those closely emulated trials showed higher agreement with the original RCT results. A post‑hoc analysis limited to closely emulated trials pushed correlation values into the 0.90s, Bradley said.
But agreement varied. Divergence arose from emulation choices and data limitations: claims data often lack biomarkers, imaging, detailed clinical inclusion/exclusion information and consistent lab results, which can force analysts to approximate trial criteria. Differences in real‑world adherence, treatment augmentation, loss to follow‑up from insurance turnover, and unmeasured confounding also contributed to mismatches.
Bradley cautioned that apparent agreement can sometimes reflect canceling effects of multiple biases rather than true concordance. She urged systematic evaluation of both concordant and discordant pairs before relying on RWD for high‑stakes regulatory decisions.
Prospective prediction and calibration
In a second part of the project, the team prospectively emulated seven ongoing phase‑4 trials before their results were posted and reported many close agreements (Bradley gave one example where an RCT estimate of 0.86 corresponded to 0.85 in the emulation). The third part, BenchXCal (benchmark, expand, calibrate), uses an initial emulation to quantify divergence, adjust parameters, then expand to related questions and perform sensitivity analyses to assess robustness.
Bottom line
Bradley said RWD emulation can add value when the trial’s design elements can be closely matched in the data, but stressed that “one size does not fit all.” She recommended early consultation with FDA, rigorous preregistration of protocols, access to patient‑level data for traceability, and careful, context‑specific evaluation before treating RWD as equivalent to randomized evidence.
The presentation included detailed examples and suggested follow‑up through the CDER RWE mailbox for sponsors or researchers seeking guidance.