An FDA clearance sounds like proof that a medical tool works. A new study suggests it often is not. Researchers at the University of Toronto, MIT Critical Data, and other groups looked at 1,357 AI medical devices that the US Food and Drug Administration had cleared or approved through December 5, 2025. Only three had been tested for patient outcomes, such as whether people lived longer, avoided serious illness, or stayed out of the hospital. Other experts reviewed the study before it was published in PLOS Digital Health on August 19, 2026.
That does not mean the other devices are useless. It means that for most of them, nobody has publicly shown whether patients do better. "Cleared" and "proven" are two different things.
What the Study Found
The team started with the FDA's public list of AI devices and a catalogue kept by the American College of Radiology. Then it searched ClinicalTrials.gov, a public registry of studies, and PubMed, a database of medical papers, to see what testing each device had.
The numbers drop at every step. Only 34 devices (2.5%) were linked to a registered trial that was planned in advance. Twelve posted results. Twelve had papers in medical journals. Just three (0.2%) looked at patient outcomes.
Most of the list is radiology, which means scans and images. That is 1,059 of the 1,357 devices, or 78%. Fewer than 1% of those had a registered trial. The paper counts three.
The trials that do exist have limits. About 62% were observational studies, which watch patients without a test group. Nearly three-quarters had fewer than 500 people. Companies ran 32 of the 34. Most took place only in the US, and many left out groups such as pregnant women and people who do not speak English.
Why Clearance Is Not Proof
Most AI devices reach the market through a route called 510(k). The FDA asks one main question: is the new device similar enough to one that is already on the market? It does not require a trial showing that patients get better. The authors say this lets weak evidence carry forward, because many of the older devices were never tested on patients either.
Drugs face a tougher test. They need clinical trials before approval. The highest-risk devices, which use a separate route, must also show clinical evidence.
The FDA's approach has a defense. In a 2026 paper, FDA scientists, including Jana Delfino, explain how the agency reviews imaging devices. They say that testing a general-purpose device on every possible clinical task would be too much work to be practical. They add that technical image-quality tests can stand in for some clinical testing. They cite the FDA's "least burdensome" guidance, which tells the agency to ask for no more evidence than it needs. The paper says it reflects the authors' views, not official FDA policy.
Accuracy tests also matter. They just answer a different question from "do patients do better?"
What the Study Cannot Tell You
The study counts only public evidence. It could miss tests that companies ran but never registered or published. It also did not compare AI devices with other medical devices, so the gap may not be unique to AI. The authors add that the FDA keeps watching devices after they go on sale, so missing public evidence does not prove weak oversight.
The numbers are also a snapshot. The study stops at December 5, 2025. The FDA's list has added devices since then. Its September 2026 update shows decisions through late June 2026. The three-device figure only covers the 1,357 the authors checked.
Why It Matters and What Could Come Next
For hospitals, clearance should not be the end of the question. A physician-focused summary of the study says to ask what prospective evidence supports a tool, as well as whether the FDA cleared it. The authors also cite a 2025 study in JAMA Health Forum by Lee and colleagues. Of 950 AI devices, 60 (6.3%) had at least one recall. About 43% of the recalls came within a year of clearance. Devices with no reported clinical validation averaged more recalls, 3.4 each, than validated ones, about 2 each.
The effect reaches beyond the US. The authors warn that FDA clearance often works like a passport for selling devices abroad, including in poorer countries that may not be able to test them locally.
In the near term, health systems can ask vendors for outcome data. The authors also suggest three steps: stronger checks on diverse data before clearance, studies of at least 500 patients around clearance, and outcome trials of at least 2,000 patients afterward. These are only proposals. The study does not say the FDA has adopted them.
What comes later is speculation. Tougher rules could slow new clearances, or push companies to run outcome trials earlier. Until that is clear, "FDA-cleared" tells you a tool met a regulatory bar, not that it helps patients.