HAI Weekly Seminar with Art Owen
Variable Importance, Cohort Shapley Value, and Redlining
Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.
Sign Up For Latest News
Variable Importance, Cohort Shapley Value, and Redlining
In the last year, AI models have rapidly improved on public benchmarks measuring their ability to carry out research tasks autonomously. Such performance suggests that models may be capable of automating parts of the AI R&D process itself, thereby accelerating the pace of AI progress.

In the last year, AI models have rapidly improved on public benchmarks measuring their ability to carry out research tasks autonomously. Such performance suggests that models may be capable of automating parts of the AI R&D process itself, thereby accelerating the pace of AI progress.
This talk will cover key insights from the third AI100 report, AI Goes Mainstream: The Scaling Era and the Rise of Generative AI, set for release in November 2026. The report was commissioned by the AI100 standing committee and led by study panel chair Michael Wooldridge.

This talk will cover key insights from the third AI100 report, AI Goes Mainstream: The Scaling Era and the Rise of Generative AI, set for release in November 2026. The report was commissioned by the AI100 standing committee and led by study panel chair Michael Wooldridge.
Sessions run Wednesdays from 4:30–5:30 PM in CoDa E160. Each session features a different Stanford speaker; talk titles are announced by the organizers.

Sessions run Wednesdays from 4:30–5:30 PM in CoDa E160. Each session features a different Stanford speaker; talk titles are announced by the organizers.
In order to explain what a black box algorithm does, Owen says, we can start by studying which variables are important for its decisions. Variable importance is studied by making hypothetical changes to predictor variables. Changing parameters one at a time can produce input combinations that are outliers or very unlikely. They can be physically impossible, or even logically impossible. It is problematic to base an explanation on outputs corresponding to impossible inputs. Owen introduces the cohort Shapley (CS) measure to avoid this problem, based on Shapley value from cooperative game theory.
There are many tradeoffs in picking a variable importance measure, so CS is not the unique reasonable choice. One interesting property of CS is that it can detect 'redlining', meaning the impact of a protected variable on an algorithm's output when that algorithm was trained without the protected variable.