Using Algorithm Audits to Understand AI

Key Takeaways

• We identified nine considerations for algorithm auditing, including legal and ethical risks, factors of discrimination and bias, and conducting audits continuously so as to not capture just one moment in time. 

• We found that researchers are activists—working on topics with social and political impacts, and behaving as actors with sociopolitical effects—and must factor the social impact of algorithmic development into their work. 

• Algorithm auditors must collaborate with other experts and stakeholders, including social scientists, lawyers, ethicists, and the users of algorithmic systems to more comprehensively and ethically understand the impacts of those systems on individuals and society at large.

Executive Summary

Artificial Intelligence continues to proliferate, from government services and academic research to the transportation, energy, and healthcare sectors. Yet one of the greatest challenges in using, understanding, and regulating AI persists: the black-box nature of many algorithms.

Dr. Latanya Sweeney’s 2013 paper, “Discrimination in Online Ad Delivery,” speaks to this very point. Sweeney, a professor at Harvard, surveyed 2,184 racially associated names in relation to searches tied to Google AdSense, Google’s service for placing ads at the top of users’ search results pages. All told, she found that ads placed on the page were far more likely to suggest an arrest record under queries for Black-sounding names than white-sounding ones—“raising questions as to whether Google’s advertising technology exposes racial bias in society and how ad and search technology can help develop to assure racial fairness.”

This question of racist or otherwise discriminatory AI is not just a widespread problem—as much other research has uncovered—it is also an issue of blackbox decision-making. With respect to Sweeney’s findings, one possibility is that Google deliberately targeted minority-sounding names with racist suggestions for “arrest records.” It is also possible, however, that internet users were more likely to search Black names and then click on websites mentioning arrest. The harms and the dangers of this algorithmic discrimination are clear, but understanding an algorithm’s decision-making process can be far more difficult. Doing so matters greatly for researchers, policymakers, and the public.

In our paper, titled “Auditing Algorithms: Understanding Algorithmic Systems from the Outside In,” we examine how algorithm audits—like the input- and output-testing Sweeney did for her research—are a powerful technique for understanding AI. In collaboration with researchers from Northeastern University, University of Illinois at Urbana-Champaign, and University of Michigan, we provide an overview of methodologies for algorithm audits, recount two decades of algorithm audits across numerous domains (from health to politics), and propose a set of best practices for conducting algorithm audits. We conclude with a discussion of algorithm audits and their social, ethical, and political implications.