AI Audit Challenge

After careful examination by the independent jury, we have made the decision to recognize four submissions with horizontal awards and highlight the unique characteristics of the projects that stood out.

• Award for the Greatest Potential: Auditbot

  The tool that audits third party ML APIs was built and open-sourced by marrying two different ideas together: (1) system status pages, which Engineers use to monitor the uptime of APIs, and (2) model cards, that ML Researchers use to report on the performance of ML models. The result is an AI audit bot: it runs once a week and uses an open dataset to evaluate the performance of Google’s sentiment API and publishes its results online as a timeline of metrics. Extending this type of prototype to cover a wide range of proprietary AI APIs can give researchers and practitioners valuable data and alerts about how the models behind them are changing over time.

  Name	Affiliation
  Neal Lathia	Monzo Bank, UK

• Award for Most Promising for Auditing LLMs: Ceteris Paribus

  Ceteris Paribus is a tool for on-the-fly, interactive, and personalized discrimination auditing of large language models. Our user-friendly solution allows users with no computational expertise to run audits that are fully customized to their needs and created from scratch.

  An auditor simply has to indicate their topic of interest and a list of protected groups. Without any pre-existing data, our tool potentially finds countless examples of the language model being discriminatory to those protected groups on the topic of interest. The auditor is intermittently asked to guide the audit in the direction they would like. They can select examples they find interesting and cross out those that are off-topic, to push the tool to find more of the former and less of the latter.

  Our hybrid approach allows us to merge the benefits of precise human judgment and large-scale automation. The auditor in the loop ensures that the audit is grounded in prompts that adhere to the standards of the organization requiring the audit, such as legal firms or policymakers, while automation ensures that endorsed examples can be rapidly expanded upon and that the landscape of discrimination is exhaustively explored, surfacing examples of discrimination that even the auditor would not have anticipated. Our ecosystem of widgets automates the manually-tedious extraction of coarse-grained discrimination trends from individual examples.

  Overall, Ceteris Paribus empowers users to take a proactive approach to discovering discrimination in language models by providing a simple and effective solution that can be tailored to the needs of any individual or organization.

  Name	Affiliation
  Adarsh Jeewajee (Co-Lead)	Stanford University
  Edward Chen (Co-Lead)	Stanford University
  Xuechen Li	Stanford University
  Ransalu Senanayake	Stanford University
  Mert Yuksekgonul	Stanford University
  Carlos Guestrin	Stanford University

• Award for Most Innovative and Empowering to the Public: End-User Audits

  Audits have risen as powerful tools to hold algorithmic systems accountable. But because AI audits are conducted by technical experts, audits are necessarily limited to the hypotheses that experts think to test. End users hold the promise to expand this purview, as they inhabit spaces and witness algorithmic impacts that auditors do not. In pursuit of this goal, we propose end-user audits—system-scale audits led by non-technical users—and present an approach that supports end users in hypothesis generation, evidence identification, and results communication. Today, performing a system-scale audit requires substantial user effort to label thousands of system outputs, so we introduce a collaborative filtering technique that leverages the algorithmic system's own disaggregated training data to project from a small number of end-user labels onto the full test set. Our end-user auditing tool, IndieLabel, employs these predicted labels so that users can rapidly explore where their opinions diverge from the algorithmic system's outputs. By highlighting topic areas where the system is underperforming for the user and surfacing sets of likely error cases, the tool guides the user in authoring an audit report. In an evaluation of end-user audits on a popular comment toxicity model with non-technical users, participants both replicated issues that formal audits had previously identified and also raised previously underreported issues such as under-flagging on veiled forms of hate that perpetuate stigma and over-flagging of slurs that have been reclaimed by marginalized communities.

  Name	Affiliation
  Michelle S. Lam (Lead)	Stanford University
  Mitchell L. Gordon	Stanford University
  Danaë Metaxa	University of Pennsylvania
  Jeffrey T. Hancock	Stanford University
  James A. Landay	Stanford University
  Michael S. Bernstein	Stanford University

• Award for Best Holistic Evaluation and Benchmarking: HateCheck

  Hate speech detection models play an important role in protecting people online. When they work well, they keep online communities safe by making content moderation more effective and efficient. When they fail, however, they create serious harms, as some people are exposed to hate that is left undetected while others are restricted in their freedom of expression when innocuous content is removed. These issues are exacerbated by biases in model performance, where hate against some groups is detected less reliably than hate against other groups.

  Our project, HateCheck, is a multilingual auditing tool for hate speech detection models. Typically, these models have been tested on held-out test sets using summary metrics like accuracy. This makes it difficult to pinpoint model weak spots. Instead, HateCheck introduces functional tests for hate speech detection models, to enable more targeted diagnostics of potential model weaknesses and biases. Each functional test contains test cases for different types of hate and challenging non-hate, including special cases for emoji-based hate. In total, HateCheck has 36,000+ test cases across 11 languages, with further expansions planned.

  Our work on HateCheck has been published at top NLP conferences, and covered by major media outlets. HateCheck is a substantial innovation in how hate speech detection models are evaluated, which supports the creation of fairer and more effective hate speech detection.

  For more info, visit www.hatecheck.ai

  Name	Affiliation
  Paul Röttger	University of Oxford
  Hannah Rose Kirk	University of Oxford
  Bertie Vidgen	University of Oxford