Response to FDA's Request for Comment on AI-Enabled Medical Devices
Stanford scholars respond to a federal RFC on evaluating AI-enabled medical devices, recommending policy interventions to help mitigate the harms of AI-powered chatbots used as therapists.
In collaboration with
Executive Summary
In this response to a request for public comment issued by the Food and Drug Administration on measuring and evaluating AI-enabled medical device performance in the real world, scholars from Stanford HAI, the Behavioral Science & Policy Institute at the University of Texas at Austin, and Carnegie Mellon University provide recommendations that focus on policy interventions to help mitigate the harms of AI-powered chatbots used as therapists or for other psychological purposes. The recommendations focus on closing the gap in systematic evaluations of chatbot efficacy and grounding design choices in behavioral sciences. They include calls to:
Develop comprehensive benchmarks that incorporate human clinical expertise.
Require chatbot developers to provide API end points for user-facing models.
Institute reporting requirements for performance evaluations and safety protocols.
Designate a trusted third-party evaluator for AI mental health chatbots.
Mandate companies to designate products designed for therapeutic uses.
Ensure chatbot design that prevents AI sycophancy and parasocial relationships.
Related Publications
Implementing the Transparency in Frontier AI Act Report for the California Department of Technology Multistakeholder Workshops
In this paper, HAI Associate Director and Senior Fellow Rob Reich and UC Berkeley School of Information Professor Deirdre K. Mulligan provide concrete recommendations to the California Department of Technology on the implementation of its frontier AI transparency requirements.
In this paper, HAI Associate Director and Senior Fellow Rob Reich and UC Berkeley School of Information Professor Deirdre K. Mulligan provide concrete recommendations to the California Department of Technology on the implementation of its frontier AI transparency requirements.
Designing Loyalty: AI Agents and Conflicts of Interest
This brief examines the conflict-of-interest risks posed by AI agents and calls for imposing a duty of loyalty on developers and deployers.
This brief examines the conflict-of-interest risks posed by AI agents and calls for imposing a duty of loyalty on developers and deployers.
Regulating Data Brokers in the Age of AI: A California Case Study
This brief assesses data broker compliance with California privacy laws and calls for more stringent consumer protections.
This brief assesses data broker compliance with California privacy laws and calls for more stringent consumer protections.
This brief highlights the emergence of world models and outlines a first-of-its-kind governance and policy agenda for the technology.
This brief highlights the emergence of world models and outlines a first-of-its-kind governance and policy agenda for the technology.