Response to FDA's Request for Comment on AI-Enabled Medical Devices
Stanford scholars respond to a federal RFC on evaluating AI-enabled medical devices, recommending policy interventions to help mitigate the harms of AI-powered chatbots used as therapists.
In collaboration with
Executive Summary
In this response to a request for public comment issued by the Food and Drug Administration on measuring and evaluating AI-enabled medical device performance in the real world, scholars from Stanford HAI, the Behavioral Science & Policy Institute at the University of Texas at Austin, and Carnegie Mellon University provide recommendations that focus on policy interventions to help mitigate the harms of AI-powered chatbots used as therapists or for other psychological purposes. The recommendations focus on closing the gap in systematic evaluations of chatbot efficacy and grounding design choices in behavioral sciences. They include calls to:
Develop comprehensive benchmarks that incorporate human clinical expertise.
Require chatbot developers to provide API end points for user-facing models.
Institute reporting requirements for performance evaluations and safety protocols.
Designate a trusted third-party evaluator for AI mental health chatbots.
Mandate companies to designate products designed for therapeutic uses.
Ensure chatbot design that prevents AI sycophancy and parasocial relationships.
Related Publications
Response to OSTP's Request for Information on Accelerating the American Scientific Enterprise
Stanford scholars respond to a federal RFI on scientific discovery, calling for the government to support a new “team science” academic research model for AI-enabled discovery.
Stanford scholars respond to a federal RFI on scientific discovery, calling for the government to support a new “team science” academic research model for AI-enabled discovery.
Russ Altman’s Testimony Before the U.S. Senate Committee on Health, Education, Labor, and Pensions
In this testimony presented to the U.S. Senate Committee on Health, Education, Labor, and Pensions hearing titled “AI’s Potential to Support Patients, Workers, Children, and Families,” Russ Altman highlights opportunities for congressional support to make AI applications for patient care and drug discovery stronger, safer, and human-centered.
In this testimony presented to the U.S. Senate Committee on Health, Education, Labor, and Pensions hearing titled “AI’s Potential to Support Patients, Workers, Children, and Families,” Russ Altman highlights opportunities for congressional support to make AI applications for patient care and drug discovery stronger, safer, and human-centered.
Michelle M. Mello's Testimony Before the U.S. House Committee on Energy and Commerce Health Subcommittee
In this testimony presented to the U.S. House Committee on Energy and Commerce’s Subcommittee on Health hearing titled “Examining Opportunities to Advance American Health Care through the Use of Artificial Intelligence Technologies,” Michelle M. Mello calls for policy changes that will promote effective integration of AI tools into healthcare by strengthening trust.
In this testimony presented to the U.S. House Committee on Energy and Commerce’s Subcommittee on Health hearing titled “Examining Opportunities to Advance American Health Care through the Use of Artificial Intelligence Technologies,” Michelle M. Mello calls for policy changes that will promote effective integration of AI tools into healthcare by strengthening trust.
Increasing Fairness in Medicare Payment Algorithms
This brief introduces two algorithms that can promote fairer Medicare Advantage spending for minority populations.
This brief introduces two algorithms that can promote fairer Medicare Advantage spending for minority populations.