Toward Responsible AI in Health Insurance Decision-Making

Key Takeaways

• Health insurers have attracted public controversy and policy attention amid reports that their use of AI tools may be contributing to wrongful coverage denials.

• We examine why health insurance utilization review processes, especially prior authorization, have become a focal point for AI adoption by both insurers and healthcare providers and why ensuring responsible deployment is challenging.

• AI has the potential to meaningfully reduce administrative burden and care delays by automating clearly approvable requests, improving documentation quality, and supporting appeals.

• However, AI tools can also exacerbate existing flaws in already fraught processes or introduce new harms — for example, by reinforcing historically unjust denial patterns.

• Insurers and providers must adopt stronger institutional governance mechanisms for vetting and monitoring AI tools to ensure they improve access to care rather than entrenching incentives to deny or delay treatment.

Executive Summary

Health insurers and healthcare providers are rapidly adopting AI tools to process prior authorization requests and adjudicate claims. According to one 2024 survey, 84% of large health insurers in 16 states were using AI for some operational purposes. This rapid uptake has been spurred by the hope that AI will streamline resource-intensive tasks like utilization review, reduce errors, and allow human reviewers to focus on more complex cases. 

However, the widespread adoption of AI in health insurance processes has also caused public controversy and attracted policy attention, amid numerous reports of AI tool usage leading to wrongful claim denials. Prior authorization, in particular, has long been plagued by delays and wrongful coverage denials, and one fear is that by making prior authorization reviews cheaper to conduct, AI could supercharge a flawed process. Many insurers do not document the accuracy of the models they deploy or test them for biases. And many have not instituted governance mechanisms to ensure accountability.

In our paper “The AI Arms Race in Health Insurance Utilization Review,” we explore why utilization review has become such a hot spot for AI applications and why ensuring responsible deployment remains challenging. Our analysis draws on empirical research into how insurers and healthcare providers use AI, including our own ethical evaluations of provider-facing tools within Stanford Health Care, a multi-hospital health system. We offer five recommendations for policymakers and healthcare organizations to consider as they decide what role AI will play in their future operations. 

AI has the potential to dramatically streamline workflows in a field burdened by high administrative costs, wrongful claim denials, and worker burnout. Yet without safeguards, AI risks reinforcing existing incentives to delay or deny care.

Introduction

Health insurers and healthcare providers have been adopting AI tools so rapidly that it has been likened to an AI arms race. A 2024 survey of large health insurers conducted by the National Association of Insurance Commissioners found that 37% reported using AI for prior authorization, 44% for claims adjudication, and 56% for utilization management activities broadly defined. 

Our own review of online offerings by AI vendors reveals a robust marketplace of generative and predictive AI tools that predominantly target either insurers or providers — though some emerging vendors sell collaborative solutions to both insurers and providers. Those products aim to bridge the payer-provider divide by standardizing data exchanges and creating shared decision frameworks.

Most commonly, AI developers market their tools to insurers to help conduct utilization review — the process insurers use to decide whether to approve payment for services recommended by an enrollee’s physician. In this context, AI is primarily used to support prior authorization (the pre-approval of treatments); concurrent review (assessing the ongoing need for care); and decisions about claims after services have been provided. Insurer-facing AI tools often determine whether a patient meets prior authorization requirements and generate related recommendations and correspondence. 

Beyond utilization review, AI tools can also support insurers with fraud detection, disease management, pricing, marketing, and risk adjustment. AI tools geared toward healthcare providers primarily aim to help providers secure prior authorization and payment of claims by gathering clinical documentation and filling out insurance forms.