A new study finds that large language models used widely for medical assessments cannot back up claims.
As the development and adoption of AI-enabled healthcare continue to accelerate, regulators and researchers are beginning to confront oversight concerns in the clinical evaluation process that could yield negative consequences on patient health if left unchecked. Since 2015, the United States Food and Drug Administration (FDA) has evaluated and granted clearance for over 100 AI-based medical devices using a fairly rudimentary evaluation process that is in dire need of improvement as these evaluations have not been adapted to address the unique concerns surrounding AI. This brief examined this evaluation process and analyzed how devices were evaluated before approval.