Response to FDA’s Request for Feedback on Considerations for the Regulation of GenAI-Enabled Medical Devices

Executive Summary

In this response to a discussion paper and request for feedback issued by the Food and Drug Administration (FDA), scholars from Stanford HAI and Stanford University’s School of Medicine share reflections on the FDA’s considerations for the regulation of generative AI-enabled medical devices. Their recommendations focus on challenges related to operationalizing the two-axis risk assessment framework and ensuring that the proposed competency-based approach to evaluation and monitoring can both maximize patient safety and support innovation. These include: 

On the Two-Axis Risk Framework 

1. Require evidence that human oversight is an effective risk control for the task(s) the device performs.

2. Specify how the consequences axis is determined and consider measures of typical use.

On Premarket Evaluation 

3. Strengthen benchmarking to address construct validity and prevent metric optimization at the expense of utility.

4. Establish real-world prospective testing as a supplement or alternative to benchmarking.

On Postmarket Evaluation 

5. Place greater accountability for postmarket monitoring on developers and equip healthcare institutions with the ability to support continued monitoring.

6. Use hybrid monitoring, with machine-based monitoring tools to flag issues and humans as the final arbiters.

7. Ensure that postmarket monitoring occurs by requiring a prespecified monitoring plan.