Advancing AI Audit - Project Showcase and Lessons Learned

Overview

How can we effectively design and develop practical tools to assess the presence of bias and potential discrimination in AI systems? Last August, HAI and Cyber Policy Center launched the AI Audit Challenge, an initiative that invited teams from around the world to submit their models, solutions, and tools with the goal of improving our ability to evaluate AI systems. This event spotlights the four award-winning entries, featuring insightful discussions from participants and AI experts on effective practices and valuable lessons learned from the Challenge. 

See the Challenge’s 2023 awards and finalists here. 

Watch the Recording

Event Organizers

Rumman Chowdhury

Responsible AI Fellow, Harvard Berkman Klein Center for Internet & Society

Marietje Schaake

International Policy Fellow, Stanford HAI; International Policy Director, Stanford Cyber Policy Center

Daniel Zhang

Senior Manager for Policy Initiatives

Agenda

9:00a.m. - 9:05a.m. PDT Opening Remarks

 

 

SpeakersMarietje Schaake HAI International Policy Fellow; International Policy Director, Cyber Policy Center, Stanford UniversityRumman Chowdhury Responsible AI Fellow, Berkman Klein Center for Internet & Society, Harvard University

Section 1: Project Showcase and Discussion

9:05a.m. - 9:15a.m. PDT Award for the Greatest Potential

Auditbot

SpeakersNeal Lathia Monzo Bank, UK 9:15a.m. - 9:25a.m. PDT Award for Most Promising for Auditing LLMs

Ceteris Paribus

SpeakersEdward Chen Stanford University 9:25a.m. - 9:35a.m. PDT Award for Most Innovative and Empowering to the Public

End-User Audits

SpeakersMichelle S. Lam Stanford University 9:35a.m. - 9:45a.m. PDT Award for Best Holistic Evaluation and Benchmarking

HateCheck

SpeakersPaul Röttger University of OxfordHannah Rose Kirk University of OxfordBertie Vidgen University of Oxford 9:45a.m. - 10:15a.m. PDT Panel Discussion

Moderated panel discussion

ModeratorsMarietje Schaake HAI International Policy Fellow; International Policy Director, Cyber Policy Center, Stanford University

Section 2: Best Practices and Lessons Learned

10:15a.m. - 11:15a.m. PDT Panel Discussion

Moderated panel discussion

SpeakersAbhishek Gupta Founder and Principal Researcher, Montreal AI Ethics InstituteGillian Hadfield Schwartz Reisman Chair in Technology and Society, Professor of Law and Professor of Strategic Management. CIFAR AI Chair. Director, Schwartz Reisman Institute for Technology and SocietyWilliam Issac Staff Research Scientist, DeepMind's Ethics and Society TeamDanaë Metaxa Raj & Neera Singh Assistant Professor, Computer & Information Science, University of Pennsylvania ModeratorsRumman Chowdhury Responsible AI Fellow, Berkman Klein Center for Internet & Society, Harvard University