Ram Shankar Siva Kumar | A Few Useful Lessons about AI Red Teaming

Abstract:

AI red teaming is exploding in popularity: At DEF CON this year, more than 2,500 hackers descended to red-team AI systems. Every organization investing in AI—from Microsoft to Google to Meta to NVIDIA—has AI red teams to actively secure their AI systems. But what does it even mean to red-team AI systems? Grounded in case studies from Microsoft, Siva Kumar contextualizes how red-teaming AI systems differs from red-teaming traditional software systems, discusses how it intersects with previous lines of inquiry such as adversarial examples, and distills eight lessons from a practitioner’s perspective.