HAI Weekly Seminar with David Robinson - Governing an Algorithm in the Wild

Biography: David Robinson is a Visiting Scientist at the AI Policy and Practice Initiative at Cornell University's College of Computing and Information Science. He is also co-founder and Managing Director of Upturn, a nonprofit that advances equity and justice in the design, governance, and use of digital technology, and a co-director of the MacArthur Foundation's Pretrial Risk Management Project. His research spans law, policy and computer science. While working at Upturn, he designed and taught a Georgetown Law seminar course on Governing Automated Decisions. David served as the inaugural Associate Director of Princeton University's Center for Information Technology Policy. He holds a JD from Yale and studied philosophy at Princeton and Oxford, where he was a Rhodes scholar.

Abstract: On December 4, 2014, the algorithm that allocates kidneys for transplant in the United States was replaced, following more than a decade of debate and planning. This process embodied many of the strategies now being proposed and debated in the largely theoretical scholarly literature on algorithmic governance (and in a growing number of legislative and policy contexts), offering a rare chance to see such tools in action. The kidney allocation algorithm has long been governed by a collaborative multistakeholder process; its logic and detailed data about its operations are public and widely scrutinized; the design process carefully assesses a complex blend of medical, moral and logistical factors; and independent experts simulate possible changes and analyze system performance. In short, a suite of careful governance practices for an algorithm operate in concert. In this talk, I reconstruct the story of the allocation algorithm’s governance and of its bitterly contested redesign, and ask what we might learn from it. I find that kidney allocation provides both an encouraging precedent and a cautionary tale for recently proposed governance strategies for algorithms. First, stakeholder input mechanisms can indeed be valuable, but they are critically constrained by existing legal and political authorities. Second, transparency benefits experts most, and official disclosures are no substitute for firsthand knowledge of how a system works. Third, the design of an algorithm allocates attention, bringing some normative questions into clear focus while obscuring others. Fourth and finally, a public infrastructure pof analysis and evaluation is powerfully helpful for informed governance.