Stanford
University
  • Stanford Home
  • Maps & Directions
  • Search Stanford
  • Emergency Info
  • Terms of Use
  • Privacy
  • Copyright
  • Trademarks
  • Non-Discrimination
  • Accessibility
© Stanford University.  Stanford, California 94305.
Navigate
  • About
  • Events
  • AI Glossary
  • Careers
  • Search
Participate
  • Get Involved
  • Support HAI
  • Contact Us
Skip to content
  • About

    • About
    • People
    • Get Involved with HAI
    • Support HAI
    • Subscribe to Email
  • Research

    • Research
    • Research Programs
    • Grants
    • Marlowe (opens in new tab)
    • Student Affinity Groups
    • Centers & Labs
    • Research Publications
    • Research Partners
  • Education

    • Education
    • Executive and Professional Education
    • Government and Policymakers
    • K-12
    • Stanford Students
  • Policy

    • Policy
    • Policy Publications
    • Policymaker Education
    • Student Opportunities
  • AI Index

    • AI Index
    • AI Index Report
    • Global Vibrancy Tool
    • People
  • News
  • Events
  • Industry
  • Centers & Labs

Stay Up To Date

Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.

Sign Up For Latest News

A Blueprint for Evaluating AI in Public Administration | Stanford HAI
policyIssue Brief

A Blueprint for Evaluating AI in Public Administration

Date
November 11, 2026
Topics
Government, Public Administration
Privacy, Safety, Security
Regulation, Policy, Governance
Read Paper
abstract

This brief presents a blueprint for how government agencies can effectively evaluate AI tools in high-stakes public services.

In collaboration with

Key Takeaways

  • Common AI evaluation methods are insufficient to help governments evaluate AI tools in practice. Despite existing legal mandates, government agencies often struggle to implement effective AI evaluation processes.

  • Agencies already possess “hidden benchmarks” — administrative records of past task performance that can be used to train and evaluate AI models to meet public sector needs. If released, such benchmarks could both measure and drive AI progress.

  • Agencies should require third-party evaluations as a condition for procurement and conduct randomized AB tests to understand AI impacts as systems are deployed. Collaborating with universities and other research institutions can offer a cost-effective way to draw on independent outside expertise.

  • Effective post-monitoring deployment will require rethinking traditional quality assurance approaches to include human-AI collaboration on oversight.

The Problem

In June 2026, autonomous AI agents gained unauthorized access to the Australian Medicare statistics portal, marking the first-known instance of an AI agent hacking a government website. This, and other recent cybersecurity incidents, illustrate the propensity of AI systems to function in ways that deviate from intended behavior and the dangers of lack of oversight. They have also inspired renewed policy proposals and legislation on auditing and evaluation. In California, for example, Governor Gavin Newsom signed Executive Order N-9-26 to accelerate the creation of an independent verification organization framework to evaluate AI systems. Private industry has likewise called for embedded evaluators in an effort to pace the frontier. In response to these incidents, Nvidia CEO Jensen Huang urged frontier labs to allocate 80% of their engineering resources to verification and 20% to development, in contrast to industry norms.

AI evaluation is not a new concern; researchers and policy experts have been developing robust approaches to AI evaluation for years. However, recent incidents have brought critical questions surrounding human oversight of AI into mainstream public debate, particularly with regard to high-impact deployments. Nowhere is this concern more pressing than in the government, which is rapidly becoming one of the largest deployers of AI tools in important domains such as healthcare, education, social services, and law enforcement. In 2025 alone, 56 federal agencies reported 3,611 individual AI use cases within government operations, a 69% increase from the previous year. 

As government agencies build chatbots and policy drafting tools, and move toward integrating more agentic AI systems into public services, two categories of risk are becoming particularly salient. The first is safety risk: AI tools can be manipulated or attacked, and agentic systems with access to government data and databases can take unanticipated actions with far-reaching consequences. The second is performance risk: AI systems may fail to perform their intended functions, or fail to deliver the intended benefits after deployment.

This brief addresses how agencies can verify the reliability of an AI tool within the highly consequential setting of public administration, where tools are used to deliver public services more effectively and advance agency missions. We outline the limitations of current approaches to AI evaluation in government and present a blueprint for effective, risk-informed evaluation. Finally, we offer recommendations for how the government can play a leading role in shaping standards and practices for evaluating increasingly powerful AI systems in mission-critical areas.

Read Paper
Share
Link copied to clipboard!
Authors
  • Danielle Sutton
    Danielle Sutton
  • Vera Lummis
    Vera Lummis
  • Dan Ho headshot
    Daniel E. Ho

Related Publications

Response to FDA’s Request for Feedback on Considerations for the Regulation of GenAI-Enabled Medical Devices
Roxana Daneshjou, Debadutta (Dev) Dash, Diliana Funtarova, Curtis Langlotz, Kara Liu, David Magnus, Josh Makower, Nicole Martinez-Martin, Caroline Meinhardt, Michelle Mello, Jared Moore, Sandra Waugh Ruggles, Kevin Schulman, Nigam Shah, Artem Trotsyuk, Caroline Yee
Quick ReadOct 19, 2026
Response to Request

Stanford scholars respond to a federal request for feedback, offering policy recommendations for the regulation of generative AI-enabled medical devices.

Response to Request

Response to FDA’s Request for Feedback on Considerations for the Regulation of GenAI-Enabled Medical Devices

Roxana Daneshjou, Debadutta (Dev) Dash, Diliana Funtarova, Curtis Langlotz, Kara Liu, David Magnus, Josh Makower, Nicole Martinez-Martin, Caroline Meinhardt, Michelle Mello, Jared Moore, Sandra Waugh Ruggles, Kevin Schulman, Nigam Shah, Artem Trotsyuk, Caroline Yee
HealthcareRegulation, Policy, GovernanceGenerative AIQuick ReadOct 19

Stanford scholars respond to a federal request for feedback, offering policy recommendations for the regulation of generative AI-enabled medical devices.

Implementing the Transparency in Frontier AI Act Report for the California Department of Technology Multistakeholder Workshops
Rob Reich, Deirdre Mulligan
Quick ReadSep 25, 2026
Response to Request

In this paper, HAI Associate Director and Senior Fellow Rob Reich and UC Berkeley School of Information Professor Deirdre K. Mulligan provide concrete recommendations to the California Department of Technology on the implementation of its frontier AI transparency requirements. 

Response to Request

Implementing the Transparency in Frontier AI Act Report for the California Department of Technology Multistakeholder Workshops

Rob Reich, Deirdre Mulligan
Regulation, Policy, GovernancePrivacy, Safety, SecurityFoundation ModelsQuick ReadSep 25

In this paper, HAI Associate Director and Senior Fellow Rob Reich and UC Berkeley School of Information Professor Deirdre K. Mulligan provide concrete recommendations to the California Department of Technology on the implementation of its frontier AI transparency requirements. 

Making Local Law Legible: LLM-Assisted Detection of Discrimination
Dan Bateyko, Yasmine Mabene, Derek Ouyang, Daniel E. Ho
Quick ReadSep 08, 2026
Policy Brief

This brief demonstrates the potential of LLM-assisted review to accelerate legal reform.

Policy Brief

Making Local Law Legible: LLM-Assisted Detection of Discrimination

Dan Bateyko, Yasmine Mabene, Derek Ouyang, Daniel E. Ho
Government, Public AdministrationEthics, Equity, InclusionQuick ReadSep 08

This brief demonstrates the potential of LLM-assisted review to accelerate legal reform.

Designing Loyalty: AI Agents and Conflicts of Interest
Ella Genasci Smith, Victor Y. Wu, Jennifer King
Quick ReadAug 25, 2026
Issue Brief

This brief examines the conflict-of-interest risks posed by AI agents and calls for imposing a duty of loyalty on developers and deployers.

Issue Brief

Designing Loyalty: AI Agents and Conflicts of Interest

Ella Genasci Smith, Victor Y. Wu, Jennifer King
Privacy, Safety, SecurityRegulation, Policy, GovernanceIndustry, InnovationQuick ReadAug 25

This brief examines the conflict-of-interest risks posed by AI agents and calls for imposing a duty of loyalty on developers and deployers.