Ram Shankar Siva Kumar | A Few Useful Lessons about AI Red Teaming
AI red teaming is exploding in popularity: At DEF CON this year, more than 2,500 hackers descended to red-team AI systems. Every organization investing in AI—from Microsoft to Google to Meta to NVIDIA—has AI red teams to actively secure their AI systems
But what does it even mean to red-team AI systems? Grounded in case studies from Microsoft, Siva Kumar contextualizes how red-teaming AI systems differs from red-teaming traditional software systems, discusses how it intersects with previous lines of inquiry such as adversarial examples, and distills eight lessons from a practitioner’s perspective.
Watch Event Recording
Related Events
Zoë Hitzig | How People Use ChatGPT
Despite the rapid adoption of LLM chatbots, little is known about how they are used. We approach this question theoretically and empirically, modeling a user who chooses whether to complete a task herself, ask the chatbot for information that reduces decision noise, or delegate execution to the chatbot...
.png&w=1920&q=100)
Despite the rapid adoption of LLM chatbots, little is known about how they are used. We approach this question theoretically and empirically, modeling a user who chooses whether to complete a task herself, ask the chatbot for information that reduces decision noise, or delegate execution to the chatbot...
Hari Subramonyam | Learning by Creating: A Human-Centered Vision for AI in Education
Joel Becker | Reconciling Impressive AI Benchmark Performance with Limited Developer Productivity Impacts
AI coding agents now complete multi-hour coding benchmarks with roughly 50% reliability, yet a randomized trial found experienced open-source developers took about 19% longer when allowed frontier AI tools than when tools were disallowed...
.png&w=1920&q=100)
AI coding agents now complete multi-hour coding benchmarks with roughly 50% reliability, yet a randomized trial found experienced open-source developers took about 19% longer when allowed frontier AI tools than when tools were disallowed...