Sanmi Koyejo | Beyond Benchmarks: Building a Science of AI Measurement
The widepread deployment of AI systems in critical domains demands more rigorous approaches to evaluating their capabilities and safety.
Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.
Sign Up For Latest News
The widepread deployment of AI systems in critical domains demands more rigorous approaches to evaluating their capabilities and safety.
This talk will cover key insights from the third AI100 report, AI Goes Mainstream: The Scaling Era and the Rise of Generative AI, set for release in November 2026. The report was commissioned by the AI100 standing committee and led by study panel chair Michael Wooldridge.

This talk will cover key insights from the third AI100 report, AI Goes Mainstream: The Scaling Era and the Rise of Generative AI, set for release in November 2026. The report was commissioned by the AI100 standing committee and led by study panel chair Michael Wooldridge.
Sessions run Wednesdays from 4:30–5:30 PM in CoDa E160. Each session features a different Stanford speaker; talk titles are announced by the organizers.

Sessions run Wednesdays from 4:30–5:30 PM in CoDa E160. Each session features a different Stanford speaker; talk titles are announced by the organizers.
This talk will explore the latest advances in computational protein design and structure prediction, highlighting NVIDIA technologies such as cuEquivariance and the Proteína family of models, including the latest RFdiffusion4 (RFD4)–Proteína developments. We will discuss how accelerated geometric deep learning and generative AI are enabling scalable, high-performance protein modeling and the design of novel proteins with tailored structures and functions.
Bio:
Youssef Nashed is a Senior Solutions Architect on NVIDIA's Healthcare and Life Sciences team. He joined NVIDIA from Bristol Myers Squibb (BMS), where he was a Senior Principal Scientist focused on machine learning and computational protein design. At BMS, Youssef led a team developing and deploying AI models for antibody discovery and development. He previously served as an Assistant Professor at Northwestern University and conducted research in structural biology at SLAC National Accelerator Laboratory at Stanford. Youssef began his career in high-performance and large-scale GPU computing at Argonne National Laboratory. His background spans computer science, mathematics, structural biology, and AI-driven protein engineering.
Location: CoDa W401

This talk will explore the latest advances in computational protein design and structure prediction, highlighting NVIDIA technologies such as cuEquivariance and the Proteína family of models, including the latest RFdiffusion4 (RFD4)–Proteína developments. We will discuss how accelerated geometric deep learning and generative AI are enabling scalable, high-performance protein modeling and the design of novel proteins with tailored structures and functions.
Bio:
Youssef Nashed is a Senior Solutions Architect on NVIDIA's Healthcare and Life Sciences team. He joined NVIDIA from Bristol Myers Squibb (BMS), where he was a Senior Principal Scientist focused on machine learning and computational protein design. At BMS, Youssef led a team developing and deploying AI models for antibody discovery and development. He previously served as an Assistant Professor at Northwestern University and conducted research in structural biology at SLAC National Accelerator Laboratory at Stanford. Youssef began his career in high-performance and large-scale GPU computing at Argonne National Laboratory. His background spans computer science, mathematics, structural biology, and AI-driven protein engineering.
Location: CoDa W401
While current evaluation practices rely on static benchmarks, these methods face fundamental efficiency, reliability, and real-world relevance challenges. This talk presents a path toward a measurement framework that bridges established psychometric principles with modern AI evaluation needs. We demonstrate how techniques from Item Response Theory, amortized computation, and predictability analysis can substantially improve the rigor and efficiency of AI evaluation. Through case studies in safety assessment and capability measurement, we show how this approach can enable more reliable, scalable, and meaningful evaluation of AI systems. This work points toward a broader vision: evolving AI evaluation from a collection of benchmarks into a rigorous measurement science that can effectively guide research, deployment, and policy decisions.
