Prior Seed Grant Recipients

2020 Seed Grant Recipients

A Resounding Experience - Multisensory Feedback for Immersive Virtual Reality

Virtual Reality technology has shown great promise in visually immersing users in a variety of environments. However, these environments lack the rich auditory and haptic signals that humans expect during real-life interactions. Our long-term research goal is to create virtual experiences that provide rich multi-sensory feedback with applications in immersive journalism, storytelling, entertainment, education or e-commerce.  We will focus on the challenge of modelling how objects sound when users interact with them. While analytical models of sound exist, they assume precise knowledge of object material and shape, the environment, and user interaction. This makes it challenging to scale up an immersive virtual environment. Additionally, analytical models may be too computationally expensive for real-time feedback. These two challenges can be addressed through learning-based approaches. We propose to develop a generative audio model that is learned from a curated video dataset showing a person interacting with objects of various textures, materials, and physical properties. This model will generate realistic sounds for objects as users interact with them in a virtual environment. This approach promises to alleviate modelling challenges by leveraging data and generalising to new objects and user interactions that are not contained in the training dataset.

Name	Role	School	Department
Jeannette Bohg	PI	School of Engineering	Computer Science
Doug James	Co-PI	School of Engineering	Computer Science
Allison Okamura	Co-PI	School of Engineering	Mechanical Engineering

A Welfare-Driven Approach to Explaining AI Decisions: The Case of Algorithmic Consumer Lending

Our ability to interpret and explain algorithmic decisions is at the center of implementing AI safely and responsibly. In consumer lending, AI comes with the promise of expanding credit access to underserved populations. But machine decisions have to be transparent to detect and mitigate discrimination, assess model robustness across the business cycle, and provide legally mandated explanations to consumers applying for credit. To deploy algorithms in a sensitive area like consumer lending, we postulate that model explanations must diagnose the specific societal impact and economic consequences an AI model has. In this project, we first develop a theoretical framework that derives desired properties of explainability techniques given regulatory objectives in consumer lending. We then propose a step-by-step approach to empirically evaluate the ability of explainability techniques to assess model behavior with respect to those regulatory objectives. Third, we demonstrate the utility of our proposed framework by applying it in a study involving commercially available explainability tools that use a variety of techniques for supplying information about a model’s functioning. Through our partnership with FinRegLab, an independent, nonprofit research organization with extensive expertise in financial policy, regulation, and compliance, we ensure the inclusion of a diverse and interdisciplinary group of stakeholders in formulating research questions, incorporating current legal and regulatory conditions, conducting the evaluation, and disseminating our research findings. This effort is designed to inform and complement ongoing efforts to develop standards for the safe, fair, and responsible use of ML/AI.

Name	Role	School	Department
Laura Blattner	PI	Graduate School of Business
Jann Spiess	Co-PI	Graduate School of Business

Adaptive Human-Robot Teaming for Cooperative Transport

As robotic hardware and intelligence improve, robots are becoming more effective teammates with humans. One collaborative task is cooperatively carrying a heavy object. When multiple humans carry a large or heavy object together, they reduce the risk of individual injury and leverage cues from other teammates and the environment to work cooperatively. This cooperation is usually characterized by efficiency: moving quickly toward a goal and not using more force than necessary. The goal of this work is to make a robot an effective teammate in this task, where at least one of the other teammates are human. By having a robotic teammate, the same advantage of mitigating personal injury remains. The challenge is teaching the robot how to leverage cues from the human teammate(s) and the environment in order to carry efficiently. To accomplish this, we plan to leverage behavioral cloning with emerging techniques in machine learning to teach the robot how to be an effective teammate in this cooperative transport task.

Name	Role	School	Department
Monroe Kennedy	PI	School of Engineering	Mechanical Engineering
Dorsa Sadigh	Co-PI	School of Engineering	Computer Science

Adaptive Situated Visualization of Uncertain Information for Troubleshooting Autonomous Robots

We are increasingly surrounded by autonomous mobile robots that augment our capabilities in a variety of applications including household cleaning, package delivery, and security monitoring. These robots operate on a variety of uncertain information, including noisy sensor data and probabilistic decision making processes. In their current state, they often fail and require users with limited training to intervene and troubleshoot the problem. To help users effectively troubleshoot robots, it is important to present relevant information and its uncertainty in an interpretable form without causing information overload. We study how humans reason about spatiotemporal data under uncertainty and subsequently strategize using the information available to them. We then leverage this understanding and explore the design of adaptive interfaces that can assist users in better troubleshooting robots.

Name	Role	School	Department
Sean Follmer	PI	School of Engineering	Mechanical Engineering

AI-Based Tool for Studying Parent-Child Interactions During Early Childhood

Significant lifetime benefits in human health are conferred from exposure to responsive parenting behaviors during early childhood. After adjusting for sociodemographic factors, such early exposure is associated with positive socioemotional, cognitive, and physical health outcomes through adulthood. As a result, much early child-development research requires carefully curated video-taped observations of parent-child interaction. These studies, however, typically employ human annotators to code these video observations manually, and because the process is so time-consuming and labor intensive, study findings have been limited by small and less diverse samples. In this project, we propose to develop an AI-based tool for automated coding of video capture of parent-child interaction, in order to advance measurement detail and scale in child-development research. Once we develop the AI-based tool, we will assess its validity by conducting a longitudinal study of patterns associated with responsive parent-child interactions, using the Study of Environmental Effects on Developing LINGuistic Skills (SEEDLingS) dataset, which captures video encounters of parent-child interactions over time.

Name	Role	School	Department
Serena Yeung	PI	School of Medicine	Biomedical Data Science
Lee Sanders	Co-PI	School of Medicine	Pediatrics

AI, Assembly and Democracy

What do political micro-targeting, platform content moderation, and the use of stingray and facial recognition technologies by local police forces have in common? Each of these AI-enabled practices reveals the role of third-parties in people’s ability to come together to take action. The Digital Civil Society Lab (DCSL) will develop a path-breaking, multi-disciplinary and collaborative volume exploring how artificial intelligence impacts a cornerstone of democratic life: the people’s ability to assemble. The manuscript (tentatively titled Artificial Intelligence, Real Democracy: How AI changes people’s ability to assemble, and what it means for democracy) will be written through a workshop process designed to produce a volume organized around a few central questions and that stands as an example of collaborative research production. The research team will also draw on our Digital Assembly Research Network (DARN) - a global community of 350+ scholars, technologists, policy makers and civil society actors dedicated to addressing the ways digital systems influence our ability and right to assemble. The volume and the DARN together serve to integrate policy, practice, and scholarship.

Name	Role	School	Department
Rob Reich	PI	School of Humanities and Sciences	Political Science
Lucy Bernholz	Co-Investigator	School of Humanities and Sciences	Center on Philanthropy and Civil Society
Toussaint Nothias	Co-Investigator	School of Humanities and Sciences	Center on Philanthropy and Civil Society

An Audiovisual Analysis of Bias in the Last Decade of Cable TV News

Each day, cable TV news networks determine what information millions of Americans receive. They also set the context and tone in which that information is presented. These decisions shape public opinion and culture. In this project we aim to develop computational tools that provide greater transparency about these editorial choices. Specifically, we will use modern deep-learning based image, text, and audio processing techniques on a decade of nearly 24-7 broadcasts from CNN, MSNBC, and Fox News to identify patterns and trends in content, bias, polarization, and editorial choices.  We will also develop new systems for quickly validating the accuracy of these machine annotations.

We will work closely with media watchdogs, journalists and other researchers to perform in-depth, data-driven analyses of specific areas of bias in cable TV news coverage. These efforts will address questions like: How much screen time is given to specific topics, and how does this differ across shows or channels? What is the gender and race distribution of expert commentators asked to speak about key topics? Our goal is to discover where such biases appear and to show the potential of AI to aid large-scale quantitative analysis of the news.

Name	Role	School	Department
Kayvon Fatahalian	PI	School of Engineering	Computer Science
Maneesh Agrawala	Co-PI	School of Engineering	Computer Science
James Hamilton	Co-PI	School of Humanities and Sciences	Communication

Augmenting Human Capabilities for Detection of Catastrophic Failures

This project focuses on creating transformative algorithms, solutions, and tools for human-AI augmented control of facilities. It will fulfill a critical need for various next-generation establishments, ranging from power grids to industrial-scale factories to nuclear plants. As these grow in complexity, their control and operation exceed human operators' abilities.

For such applications, human-AI augmentation is essential. Still, pure AI approaches create additional risks, with black-box AI solutions having uncertain validity and robustness. Our research paradigm focuses on using AI-agents not to supplant but to supplement human operators. However, the introduction of a highly automated AI-agent can cause "out-of-the-loop'' syndrome, whereby operators become increasingly detached from the system and may lose their ability to act when the AI fails or becomes unavailable. Also, a critical weakness of AI prediction systems is their opacity and lack of interpretability. Consequently, our focus will be on designing an AI system that augments human operators but leaves them in charge through interpretability, transparency, and explainability.

Name	Role	School	Department
Eric Darve	PI	School of Engineering	Mechanical Engineering
Ram Rajagopal	Co-PI	School of Engineering	Civil and Environmental Engineering

Automated Disinformation: Investigating the Potential of GPT-3 in Information Operations

Recent developments in AI—including the creation of OpenAI’s GPT-3 autoregressive language model—have improved language-generating capabilities in tasks such as translation, question-answering, and content creation. Earlier generations of OpenAI’s technology, such as GPT2, have already been used to write emails to colleagues and blog posts by venture capitalists, and similar technologies may be used by malicious actors for disinformation campaigns. The extent to which GPT-3 poses a threat to U.S. public discourse depends partly on persuasiveness and detectability. In this study, we will conduct multiple survey experiments comparing the persuasiveness and perceived credibility of GPT-3 created content, content created by known disinformation actors, and content created by people sharing their authentic beliefs. We plan to vary topics, content length, and content format in order to assess the conditions under which AI-created text is likely to be effective. We will also look at the effect of learning about the abilities of GPT-3 on general trust in media. If GPT-3 can produce content that is nearly as, or more convincing than, human-created content, GPT-3 may pose a significant risk to public discourse and the quality of the information environment.

Name	Role	School	Department
Christopher Manning	Co-PI	School of Engineering	Computer Science
Michael Tomz	Co-PI	School of Humanities and Sciences	Political Science

Building Culturally-Resonant AI to Fight Affective Propaganda

According to the U.S government, the persistent spread of fake news and other types of misinformation is one of the main on-going threats to societal cohesion and trust. This misinformation is often disseminated by computational “bots” that first learn about specific social media landscapes and then tailor fake news to be “emotionally and culturally resonant” with those landscapes. Thus, while there is a clear need for digital defense tools, their effectiveness hinges on a deep understanding of why certain affective content garners increased engagement or is “culturally resonant”. Unfortunately, we know relatively little about how affective and cultural factors shape the spread of misinformation on social media. Based on our previous research (Hsu et al., 2020), we propose a values-violation account of virality, in which we predict that the affective content that is most likely to spread is that which violates dominant cultural values regarding emotion. According to this account, high arousal negative content spreads the most in the U.S. because it violates the American value placed on maximizing positive feelings and minimizing negative ones. To further test this account, we propose to: (1) build natural language processing tools to examine and compare the spread of affective content on social media, and (2) develop algorithms capable of supporting “affective filters” that can be deployed on social media platforms to flag or modify affective content, providing a culturally-adaptable defense against affectively viral misinformation. We will also examine affective virality in specific subgroups within each country. These studies will not only advance AI research by integrating machine learning algorithms with human emotion and culture, but also advance our understanding of the kinds of affective propaganda users are most vulnerable to so that organizations individuals can defend against them.

Name	Role	School	Department
Jeanne Tsai	PI	School of Humanities and Sciences	Psychology
Michael Bernstein	Co-PI	School of Engineering	Computer Science
Johannes Eichstaedt	Co-PI	School of Humanities and Sciences	Psychology
Jeffrey Hancock	Co-PI	School of Humanities and Sciences	Communication
Brian Knutson	Co-PI	School of Humanities and Sciences	Psychology

Contactless Privacy-Preserving Ambient Intelligence for In-Home Care After Hospital Discharge

Readmission of patients after discharge from hospital is common and often costly. In the United States, one-fifth of the discharged patients (nearly 2.6 million seniors), face an acute medical complication within 30 days that requires readmission or emergency department (ED) visits. In this project, we propose strategies to reduce this readmission rate, especially among older adults. We introduce an intervention approach based on the monitoring of Activities of Daily Living (ADL) trajectories using the computer vision (CV) technology.

Name	Role	School	Department
Fei-Fei Li	Co-PI	School of Engineering	Computer Science
Kevin Schulman	Co-PI	School of Medicine	Hospital Medicine
Ehsan Adeli	Co-PI	School of Medicine	Psych/Major Laboratories and Clinical & Translational Neurosciences Incubator

Designing Equitable Decision Algorithms in the Presence of Externalities

Over the last several years, researchers and practitioners have worked to design fair prediction models, used in criminal justice, health care, social services, and beyond. Less attention, though, has been paid to the inextricable link between predictions and decisions. In this project, we aim to develop methods for creating equitable prediction-decision systems that respect the often complex preferences and constraints that arise in many applied problems. We will develop our conceptual framework within the context of a high-stakes application: designing online advertising strategies to recruit participants for CalFresh, California's Supplemental Nutrition Assistance Program (SNAP), which helps people with lower incomes buy nutritious food. Through scientific advancements in fair machine learning, we have an opportunity to substantially improve food security.

Name	Role	School	Department
Sharad Goel	PI	School of Law
Susan Athey	Co-PI	Graduate School of Business

Digital Performers Using AI

With the COVID 19 pandemic, the live performance industry has been shut down.  Many audiences and performers find that Zoom performances are unsatisfying both due to the constraints of web cameras and to the slow network speed. However, there is an unexplored area of technology that could be applied to improving streamable live performances, one that utilizes machine learning algorithms to generate “virtual performers” that are controlled by a live performer. We approach working with these algorithms from a different perspective—to think about these systems of technology from a poetic and creative standpoint. Can we improve theatrical streamed performances using machine learning? What are the creative affordances that appear when we apply artificial intelligence to the representations of characters within a live performance? Our project will undertake research both in a creative field by exploring the aesthetics of a “digital performer” alongside a rigorous scientific analysis of this networking protocol and technology behind it.

Name	Role	School	Department
Michael Rau	PI	School of Humanities and Sciences	Theater and Performance Studies
Tsachy Weissman	Co-PI	School of Engineering	Electrical Engineering
Keith Winstein	Co-PI	School of Engineering	Computer Science

Early Detection of Wildfires Using Scalable AI Algorithms

Wildfires are increasing in prevalence and severity, causing incredible human suffering and damage to property. In 2009-2018, 7 million acres burned in California, double the amount from 1979-1988. To combat wildfires in the Western US, ALERTWildfire has deployed 600 remotely operable pan-tilt-zoom cameras that have the potential to enable early detection of wildfires. Unfortunately, human monitoring is too expensive for these vast networks of cameras. We will design statistical algorithms and a runtime system to aid human operators in monitoring these camera networks for early wildfire detection. Designing such algorithms and systems must address key challenges: the need for low-latency, early detection, limited resources in the form of computational resources and human attention, and constantly changing environments in nature. We hope to allow scaling to tens of thousands of cameras, enabling early responses to potentially devastating wildfires.

Name	Role	School	Department
Tatsunori Hashimoto	PI	School of Engineering	Computer Science
Trevor Hebert	Co-PI	School of Humanities and Sciences	Jasper Ridge
Matei Zaharia	Co-PI	School of Engineering	Computer Science

Explainable AI for All: A System for Studying Interpretable AI in Online Media Consumption

The role of AI in providing us content online through social media feeds and search engine results is critically important—from consumer choices to political opinions, we know that these algorithms can change our beliefs and behaviors. But they often do so in unpredictable and invisible ways; studying AI-mediated content is a complex challenge, since most of these algorithms are proprietary and inaccessible to researchers and the public, and users might interact with dozens of different systems online every day. Our team will spend the coming year building a system to help users understand the role of AI in the online content they consume, and measure the impact this content has on them, specifically in the domain of news media and political information. To do this, we are building an in-browser tool that combines a web application and browser extension to measure the algorithmic content users receive, make insights about this content visible to them, and deploy interventions and surveys to understand how this affects them. This research will highlight the way AI systems influence us through our media environments, and the impact of making that influence more transparent, with the goal of promoting the development of AI-powered online systems that are interpretable and human-centered. 

Name	Role	School	Department
Jeffrey Hancock	PI	School of Humanities and Sciences	Communication

Global Robotic Microscopic Network for Training and Automation of Malaria Diagnosis

Access to quantitative, robust, yet affordable diagnostic tools is necessary to reduce global infectious disease burden. Manual microscopy has served as a bedrock for diagnostics with wide adaptability, although at a cost of tedious labor and human errors. Automated robotic microscopes are poised to enable a new era of smart field microscopy but current platforms remain cost prohibitive and largely inflexible, especially for resource poor and field settings. Here we present Octopi, a low-cost and reconfigurable autonomous microscopy platform capable of automated slide scanning and correlated bright-field and fluorescence imaging. With roughly two orders of magnitude in cost reduction, we aim to develop an Octopi based large robotic microscope network for improved disease diagnosis while providing an avenue for collective efforts for development of modular instruments and machine learning algorithms for better disease diagnostics for the masses.

Name	Role	School	Department
Manu Prakash	PI	School of Engineering	Bioengineering

Human-Like Visual Learning with Developmentally-Appropriate Self-Supervised Deep Neural Networks

Deep neural networks have emerged as leading models for predicting neural data from a variety of brain areas and species. We will explore whether this modeling framework can be used to predict how neural representations of visual stimuli change over development. Like the human visual system, deep network models of the adult visual system are created from a combination of pre-specified “hardware” and an extensive period of exposure to visual images.  We will thus approach the modeling in two ways, one is to build models that represent the initial conditions of the visual cortex prior to the onset of visual experience and the second is to use the training phase of models that differ in their architectures and training rules as models of human brain development.  Critically, to test these models, we will acquire rich, high temporal resolution data sets from developing human infants using high-density EEG recordings.

Name	Role	School	Department
Dan Yamins	PI	School of Humanities and Sciences	Psychology
Anthony Norcia	Co-PI	School of Humanities and Sciences	Psychology
Kalanit Grill-Spector	Co-PI	School of Humanities and Sciences	Psychology

Illuminating Illegal Fishing Through Satellite Imagery and Vessel Tracking Systems

Illegal fishing is a pressing global problem that threatens food security, livelihoods, ecosystems, and robs governments and communities of billions of dollars in income. Current approaches focus on detecting illegal fishing by commercial vessels in international waters. We propose to develop an AI-based algorithm to detect fishing vessels of all sizes, using computer vision to automatically analyze satellite imagery, focusing on nearshore areas and ports that are not well-studied. By pairing these data with available GPS-based data from vessel tracking systems, we aim to paint a comprehensive picture of vessel activities in places where fisheries are central to local communities economies and well being. With this research, we aim to address key gaps in currently available data and understanding, and highlight priorities for future studies to improve fisheries transparency and eliminate illegal fishing.

Name	Role	School	Department
Fiorenza Micheli	PI	School of Humanities and Sciences	Hopkins Marine Station
Trevor Hastie	Co-PI	School of Humanities and Sciences	Statistics
Jim Leape	Co-PI		Woods Institute for the Environment
Serena Yeung	Co-PI	School of Medicine	Biomedical Data Science

Improving Medical Decision Making through Observationally Supervised Learning

Physicians face an exponential influx of medical data. The lack of computerized assistance places an immense cognitive burden on physicians who must manually assimilate, review and interpret these data to make clinical decisions leading to treatment variability and physician burnout. Artificial intelligence (AI) techniques show promise to alleviate these problems but suffer from two main limitations: (1) They require enormous quantities of labelled data for training. Curating large labeled datasets is time-consuming and expensive; and (2) training labels are often simplistic and task-oriented (e.g. normal vs. abnormal). Even state-of-the-art AI models trained on such labelled data often fail to generalize to unseen real-world patients. The goal of this project is to develop new methodology for transferring acquired human insights and knowledge to machine learning models in the context of their providing healthcare, stepping away from classic supervision using simplistic labeling or specifying explicit rules, by simply observing skilled humans in action. We introduce a novel paradigm for developing more robust AI models that we call “observational supervision,” wherein we train AI models by observing how experienced physicians examine medical data while making decisions. Our approach not only requires minimal, if any hand labelling of data, but the signals derived from physicians’ actions (e.g. what data they look at, the order in which they look, length of time, etc.) also reflect the physician’s underlying thought process that can be injected into model training. Moreover, our work will be applicable to and benefit many other domains beyond medical care, such as augmented reality - virtual reality (AR-VR), sensor fusion, environmental monitoring, among others. 

Name	Role	School	Department
Daniel Rubin	PI	School of Medicine	Biomedical Data Science
Jonathan Chen	Co-PI	School of Medicine	Biomedical Informatics
Chris Re	Co-PI	School of Engineering	Computer Science

Intelligent Hybrid Physical-Digital Spaces: Advancing the Science, Design, and Operation of Built Environments

Name	Role	School	Department
Sarah Billington	PI	School of Engineering	Civil and Environmental Engineering
James Landay	Co-PI	School of Engineering	Computer Science

Joint Human-AI Feedback for Learning Drawing

Design sketching is both a fundamental skill in artistic expression and a powerful form of visual communication in its own right. However, outside of classroom environments, tools supporting design sketching cannot yet provide feedback and critique necessary for learning. We propose a joint human-AI approach to providing student feedback on design sketching, thereby augmenting human capacities to learn drawing and design sketching. We will develop Korekta, a human+AI corrective feedback system that balances evaluation between AI---for the skills that can be classified algorithmically---and humans---for the skills that cannot. We will further demonstrate how Korekta's correction process generates training data that helps its AI improve itself as a side effect.

Name	Role	School	Department
Michael Bernstein	PI	School of Engineering	Computer Science
Camille Utterback	Co-PI	School of Humanities and Sciences	Art

Leveraging AI to Promote Anti-racism and Foster Inclusion in Online Communities

In the wake of George Floyd’s death, calls for companies to fight racism and racial bias in their organizations and products have escalated. In the tech world, machine learning algorithms that reproduce, reinforce, and even amplify racial disparities and biases in society have been identified as key sites for intervention. Our team of social psychologists, computational linguists, and computer scientists will collaborate to develop a series of AI tools that will detect and root out bias rather than magnify it.  Working with Nextdoor, a neighborhood-focused social network, we plan to pair our bias detectors with behavioral nudges and interventions that encourage inclusive rather than exclusive behavior among users on the platform. With these detectors, we will be able to experimentally test and iterate multiple interventions across a large number of users, leveraging the strengths of AI and social science to support positive behavioral change and strengthen communities.

Name	Role	School	Department
Jennifer Eberhardt	PI	School of Humanities and Sciences	Psychology
Dan Jurafsky	Co-PI	School of Humanities and Sciences	Humanities
Hazel Markus	Co-PI	School of Humanities and Sciences	Behavioral Sciences

Multimodal & Multidomain Stress Sensing

Name	Role	School	Department
Pablo Paredes Castro	PI	School of Medicine	Psych/Major Laboratories and Clinical & Translational Neurosciences Incubator
Mert Pilanci	Co-PI	School of Engineering	Electrical Engineering

Naturalization of the Laryngectomy Voice Using Audio-to-Audio Translation

One’s voice is arguably one of the most human aspects of a person. Unfortunately, patients with laryngeal cancer often undergo laryngectomies (or surgical removal of the voice organ) and consequently suffer permanent loss of their voice. Current methods for voice rehabilitation rely on modulation of artificially produced vibrations, which results in a voice that is unnatural, monotonous and difficult for an untrained listener to understand. We aim to develop a process to naturalize and individualize the post-laryngectomy voice. More specifically, our approach will be to apply machine learning methods to perform audio-to-audio translation of post-laryngectomy speech, utilizing the unique characteristics of the patient’s pre-laryngectomy voice. We anticipate that any meaningful voice naturalization will help patients regain their individuality, enhance communication, and improve overall quality of life.

Name	Role	School	Department
Fred Baik	PI	School of Medicine	Otolaryngology
Maneesh Agrawala	Co-PI	School of Engineering	Computer Science

Opening the Loop

Opening the Loop is the tentative name for a documentary film being directed and produced by Muhammad Khattak (Stanford) and Joe Khoury (California Institute of the Arts). The film seeks to critically interrogate how broader issues regarding power, culture and our ethical priorities influence current developments of artificial intelligence. Primarily focusing on AI’s regulatory uses, it aims to shift away from predominant attitudes that technology is merely a neutral tool and focuses on AI’s slow expansion into the public realm. This is part of a broader project to promote non-technical engagements in AI. In conjunction with the film, we will be collaborating with Stanford artists and educators to organize an AI art exhibit and high school outreach program about the societal implications of AI. The latter is meant to introduce ethical questions about AI to the realm of secondary education whereas the former serves to promote unique engagements in current technological changes. Together, these three projects focus on the particular experiences that are oftentimes marginalized in popular representations of AI and aim to endorse more open-ended, philosophical and artistic means of describing AI in current conversations.

Name	Role	School	Department
Russell Berman	PI	School of Humanities and Sciences	Comparative Literature
Ruth Starkman	Co-PI	School of Humanities and Sciences	Writing and Rhetoric

Technology and Racial Equity Action Lab: Linking AI Research and Practice for Racial Equity

Rapidly developing technologies that use artificial intelligence can be an unprecedented force for good; however, they can also codify and amplify existing forms of racial inequality, discrimination, and bias. The Technology and Racial Equity Action Lab, based at the Center for Comparative Studies in Race and Ethnicity, combines research, teaching, and practice to advance racial justice in the analysis, production, and deployment of new technologies. The Lab will produce public-facing reports, policy briefs, and recommendations on critical issues at the intersection of race and technology. In addition, the Lab expands these impacts through its international Practitioner Fellows Program that brings together these external fellows with Stanford students and faculty.

Name	Role	School	Department
Jennifer DeVere Brody	PI	School of Humanities and Sciences	Theater and Performance Studies
Teresa LaFramboise	Co-PI	Graduate School of Education
Michele Elam	Co-PI	School of Humanities and Sciences	English

School Choice Mechanisms: Improving Diversity and Equity

More than sixty years after “Brown v. Board of Education”, segregation by race and class is still a growing problem in public schools in the US. Many school districts adopted student assignment mechanisms in order to provide families more options while hoping also to disentangle neighborhood segregation and school segregation. Although families express their preferences in such mechanisms, priorities and zones can be designed by the district to achieve societal objectives. Ad-hoc design choices have been incorporated by districts to address these issues, such as assigning priority to students from neighborhoods with low income or a high percentage of minorities. However, there is little evidence that such policies in the context of choice have helped to address concerns about equity. In 2018, the San Francisco Unified School District passed a resolution for developing a new student assignment system for elementary schools, which seeks to improve diversity, transparency, and equal access to quality schools. This project will build tools from AI, optimization and Economics to design mechanisms, priorities and zones towards achieving these goals.

Name	Role	School	Department
Itai Ashlagi	PI	School of Engineering	Management Science and Engineering
Irene Lo	Co-PI	School of Engineering	Management Science and Engineering

SEE: The Science and Engineering of Explanations

Explanations are critical to how humans understand and learn about the world. As humans, we readily go beyond what happened to reason about why it happened and how things could have played out differently. While today’s AI systems can achieve super-human performance on many challenging tasks, their lack of generalizability, interpretability, and ability to interact with humans limits their potential, especially in high-stakes settings. One key aspect of human intelligence that is missing is the ability to understand and communicate about causality. Drawing inspiration from how humans think and communicate about causality, the Science and Engineering of Explanations (SEE) project pursues the twin goal of developing AI systems that generate and understand causal explanations the way humans do, and that help to improve human explanatory abilities.

Name	Role	School	Department
Tobias Gerstenberg	PI	School of Humanities and Sciences	Psychology
Hyowon Gweon	Co-PI	School of Humanities and Sciences	Psychology
Thomas Icard	Co-PI	School of Humanities and Sciences	Philosophy
Percy Liang	Co-PI	School of Engineering	Computer Science
Jiajun Wu	Co-PI	School of Engineering	Computer Science

Self-Modeling in Humans and Artificial Systems

Take a moment to attend to the varying degrees of control you have over your surroundings. Your limbs are decidedly part of you: you can control them directly, sensing them through high-bandwidth proprioception. Further along this "agency spectrum" are tool-like objects such as forks, which you can grab, control, and use as extensions of yourself, albeit without proprioceptive feedback. Other objects, like a table, can be manipulated, yet the act of doing so is less direct than with tools. Then there are other agents, which you can affect even less directly, such as through social interaction. Finally, there are aspects of the environment that affect you but cannot easily be affected by you, such as the weather, and yet other aspects which neither affect you nor can be affected by you, such as far-off buildings or background noise. 

We hypothesize that two major mysteries regarding agency are strongly connected. The first is computational: how do we design an artificial system with a complex, capable embodiment that learns how to efficiently leverage the full spectrum of agency available to it in a natural manner---that is, through exploration, and without explicit supervision? The second problem is cognitive: humans represent an agentic self as a distinct entity from the world. What is the utility of this self: why and how do we cognitively track it in this distinguished way? We strive to answer these computational and cognitive questions, while also elucidating the underpinnings of dissociation-- a psychiatric disruption of self implicated in prevalent stress and trauma-related mental disorders.

Name	Role	School	Department
Nick Haber	PI	Graduate School of Education
Karl Deisseroth	Co-PI	School of Engineering	Bioengineering

Self-Supervised Learning for Remote Sensing Images and Sustainability Applications

Progress on the UN Sustainable Development Goals (SDGs) is hampered by a persistent lack of data regarding key social, environmental, and economic indicators, particularly in developing countries. While there have been numerous attempts to fill this gap through the combination of machine learning and satellite imagery, a key challenge in many sustainability applications is the lack of sufficient labeled training data. In particular, geospatial analysis lacks methods like the word-vector representations and pre-trained networks that significantly boost performance across a wide range of natural language and computer vision tasks. To fill this gap, we propose to extend recent advances in contrastive self-supervised learning to remote sensing data. On natural images, these techniques can learn in an unsupervised way (i.e., without human annotations) representations that perform close to their supervised counterparts. By extending these techniques to satellite imagery, we expect to be able to learn representations that can significantly improve performance in downstream classification tasks such as poverty and food security estimation.

Name	Role	School	Department
Stefano Ermon	PI	School of Engineering	Computer Science
David Lobell	Co-PI	School of Earth, Energy and Environmental Sciences	Earth System Science

Symbols and Programs in the Visual World

Much of our visual world is highly regular: objects are often symmetric and have repetitive parts; scenes may consist of objects organized in a repetitive layout. The goal of this project is to build machines that infer and represent such regular structures from raw visual data and later leverage them for scene understanding and analysis. The key technical innovation will be methods that integrate program representations into modern deep networks.

Name	Role	School	Department
Jiajun Wu	PI	School of Engineering	Computer Science
Alex Aiken	Co-PI	School of Engineering	Computer Science

The Stanford Human Trafficking Data Lab

Stanford University´s Human Trafficking Data Lab is a multi-stakeholder effort to develop scalable technologies that leverage the modern data economy to find human trafficking and support interventions for those who have been enslaved. Our Lab is assembling powerful anti-trafficking data resources to create a unique decision support system combining near real-time machine learning insights with AI technology capable of proactively identifying forced labor and trafficking. This project will further enable us to crosslink satellite imagery with cases of illegal labor camps, producing open-source tools and a newly digitized inspection report cache, in order to disrupt systems of exploitation. 

Name	Role	School	Department
Victoria Ward	PI	School of Medicine	Pediatrics
Mike Baiocchi	Co-PI	School of Medicine	Epidemiology and Population Health
Grant Miller	Co-PI	School of Medicine	Health Policy and Primary Care Outcomes

Using Autonomous Aircraft and Uncertainty-Informed Decision Making to Reduce Uncertainty in Sea Level Rise

Sea level rise and its associated effects will be among the most immediate and dramatic impacts of climate change in this century. Small changes in mean sea level increase the rate of high sea level events that can have devastating impacts on coastal communities. Despite its importance, sea level rise projections have significant error margins, in large part due to high uncertainty in the response of Greenland and Antarctica to a changing climate. This is a result of sparse measurements, high measurement uncertainty, and incomplete understandings of the governing physics of ice sheets. Advances in autonomous aircraft promise to enable continuous data collection campaigns over ice sheets previously considered logistically impossible, but even a huge volume of data will not resolve current uncertainties if it is collected without regard for the uncertainty in the governing physics. Building on techniques from scientific machine learning and Bayesian inference, we are investigating an uncertainty-aware approach to selecting measurement locations to maximize the value of autonomous data collection aircraft in improving future sea level rise estimates.

Name	Role	School	Department
Dustin Schroeder	PI	School of Earth, Energy and Environmental Sciences	Geophysics
Mykel Kochenderfer	Co-PI	School of Engineering	Aeronautics and Astronautics

VIVA – VIdeo Visit triage using AI

During the COVID-19 pandemic, health care resources are stretched to the limit. Lack of testing capabilities and proliferation of “worried well” patients have put pressure on clinicians to prioritize needs at healthcare facilities. An efficient and effective way to triage patients has become increasingly crucial.

We aim to develop and validate a novel video-based AI algorithm to automatically triage patients with COVID-19 concerns based on a combination of their clinical appearances and their reported symptoms during a telehealth video encounter. The novel algorithm can help determine which patients need emergency assistance, which patients need to come to the clinic for further evaluation, and which patients are safe to stay home.

If successful, the novel algorithm can enable large-scale triage of patients with COVID-19 concerns without using clinicians’ time, thus using digital technologies to boost the health system's capacity. In addition, this video-based self-triage tool could potentially reduce healthcare disparity in underserved populations given their poor access to in-person providers but relatively high rate of mobile phone and internet usage. Finally, the video-based algorithm can be broadly applied to other respiratory outbreaks and diseases beyond COVID-19.

Name	Role	School	Department
Rusty Hofmann	PI	School of Medicine	Interventional Radiology
Nigam Shah	Co-PI	School of Medicine	Biomedical Informatics
Serena Yeung	Co-PI	School of Medicine	Biomedical Data Science
James Quinn	Co-PI	School of Medicine	Emergency Medicine
Wui Ip		Stanford Health Care	Pediatric Hospital Medicine
Elyse Ruan		Stanford Health Care	Digital Health Strategy

2019 HAI Seed Grant Recipients

Artificial Intelligence for Scientific Discovery

• Susan Athey (Business)
• David Hirshberg (Economics)
• Guido Imbens (Business, Economics)
• Stefan Wager (Business)

A Causal Decision-Learning Approach for Identifying Cost-Effective Clinical Pathways

• Mohsen Bayati (Business)
• Merle Ederhof (CERC)
• James Martin (Stanford Health Care)
• Alex Chin (Radiation Oncology)
• Jeffrey Jopling (Surgery)

Virtual Multisensory Interaction: From Robots to Humans

• Jeannette Bohg (Computer Science)
• Allison Okamura (Mechanical Engineering)
• Doug James (Computer Science)

Statistical Machine Learning for Understanding and Improving Social Mobility Among the Poor: A Precision Approach to Intervention

• Emma Brunskill (Computer Science)
• Geoff Cohen (Education)

Understanding and Addressing Ethical Challenges with Implementation of Machine Learning to Advance Palliative Care

• Danton Char (Anesthesiology)
• Henry Greely (Law)
• Nigam Shah (Medicine, Biomedical Data Science)

Administering by Algorithm: Artificial Intelligence in the Regulatory State

• David Freeman Engstrom (Law)
• Dan Ho (Law)
• Tino Cuellar (Law)

RoboIterum: An Augmented Reality Interface for Iterative Design of Situated Interactions with Intelligent Robots

• Sean Follmer (Mechanical Engineering)
• James Landay (Computer Science)
• Parastoo Abtahi (Computer Science)

Urbanization at the Margins: Edges & Seams in the Global South

• Roz Naylor (Earth System Science)
• Zephyr Frank (History)
• Erik Steiner (Spatial History Project)
• Deland Chan (H&S)
• Leonardo Barleta (History)

Deep Neural Network for Real-Time EEG Decoding of Musical Rhythm Imagery: Towards a Brain-Computer Interface Application for Stroke Rehabilitation

• Takako Fujioka (Music)
• Irán Román (Music, Neuroscience)
• Jay McClelland (Psychology)

Multi-modal Inference in Brains, Minds, and Machines

• Tobias Gerstenberg (Psychology)
• Justin Gardner (Psychology)
• Hyo Gweon (Psychology)
• Scott W. Linderman (Statistics, Neuroscience)

Video-based Real Time Monitoring of Respiratory Conditions in the Pediatric Emergency Department

• Sumit Bhargava (Pediatrics)
• Dan Imler (Emergency Medicine)
• Peyton Greenside (Computer Science)
• Karan Goel (Computer Science)

Learning to Play: Understanding Infant Development with Intrinsically Motivated Artificial Agents

• Daniel LK Yamins (Psychology)
• Fei-Fei Li (Computer Science)
• Mike Frank (Psychology)
• Nick Haber (Psychology)
• Damian Mrowca (Computer Science)
• Stephanie Wang (Computer Science)
• Kun Ho Kim (Computer Science)
• Eli Wang (Electrical Engineering)
• Bria Long (Psychology)
• Judy Fan (Psychology)
• George Kachergis (Psychology)
• Hyo Gweon (Psychology)

Folk Theories of AI Systems: An Approach for Developing Interpretable AI

• Jeffrey Hancock (Communication)
• Michael Bernstein (Computer Science)
• Sunny Liu (Communication)
• Danae Metaxa (Computer Science)

“Personality Design” in Artificial Intelligence-enabled Robots, Conversational Agents and Virtual Assistants

• Pamela Hinds (Management Science and Engineering)
• Angele Christine (Communication)
• Prachee Jain (Management Science and Engineering)

Using AI to Safeguard Our Drinking Water

• Kate Maher (Earth System Science)

• Jef Caers (Geological Sciences)

• Bill Mitch (Civil and Environmental Engineering)

• James Dennedy-Frank (Biology)

Predicting Learning Outcomes with Machine Teachers

• Roy Pea (Education), Emma Brunskill (Computer Science), Bethanie Maples (Computer Science), Joyce He (Education)

Developing a Computational Approach for Identifying Characteristics of Psychotherapy

• Bruce Arnow (Psychiatry and Behavioral Sciences)

• Stewart Agras (Psychiatry and Behavioral Sciences)

• Nigam Shah (Medicine, Biomedical Data Science)

• Fei-Fei Li (Computer Science)

• Adam Miner (Psychiatry and Behavioral Sciences)

• Albert Haque (Computer Science)

• Michelle Guo (Computer Science)

Crowdsourcing Concept Art: An Art Style Classifier to Maintain Consistent Artistic Vision at Scale

• Michael Bernstein (Computer Science)

• Camille Utterback (Art and Art History)

Modeling Finger Sense Training and Math Learning in Children

• Allison Okamura (Mechanical Engineering)

• Jo Boaler (Education)

• Dorsa Sadigh (Computer Science, Electrical Engineering)

• Melisa Orta Martinez (Mechanical Engineering)

• Julie Walker (Mechanical Engineering)

• Margaret Koehler (Mechanical Engineering)

PopBots: An Army of Conversational Agents for Daily Stress Management

• Dan Jurafsky (Computer Science, Linguistics)

• Pablo E. Paredes (Radiology, Psychiatry and Behavioral Sciences)

Opportunistic Screening for Coronary Artery Disease Using Artificial Intelligence

• David Maron (Cardiovascular Medicine)

• Alex Sandhu (Cardiovascular Medicine)

• Fatima Rodriguez (Cardiovascular Medicine)

• Bhavik Patel (Radiology)

• Matthew Lungren (Radiology)

• Curtis Langlotz (Radiology)

• Christopher Chute (Computer Science)

• Pranav Rajpurkar (Computer Science)

Promoting Well-Being by Predicting Behavioral Vulnerability in Real-Time

• Jeffrey Hancock (Communication)

• Gabriella Harani (Communication)

• Jure Leskovec (Computer Science)

• Adam Miner (Psychiatry and Behavioral Sciences)

• Róbert Pálovics (Computer Science)

• Katie Roehrick (Communication)

Using Artificial Intelligence to Optimize Patient Mobility and Functional Outcomes

• Arnold Milstein (Medicine)

• Fei-Fei Li (Computer Science)

• Francesca Rinaldo Salipur (CERC, General Surgery)

• Bingbin Liu (Computer Science)

Predicting Malaria Outbreaks: AI to Learn, Classify and Predict Across Diverse Paleo-demographic, Climatic and Genomic Data

• Krish Seetah (Anthropology)

• Robert Dunbar (Earth System Science)

• Carlos Bustamante (Biomedical Data Science, Genetics)

• Giulio De Leo (Biology)

• Erin Mordecai (Biology)

• Michelle Barry (CIGH)

• Bright Zhou (Medicine)

• David Pickel (Classics)

• Hannah Moots (Anthropology)

Robust Deep Neural Network Optimization with Second Order Method for Biomedical Applications

• Stephen Boyd (Electrical Engineering)

• Mohsen Bayati (Business)

• Lei Xing (Radiation Oncology)

• Varun Vasudevan (ICME)

• Kate Horst (Radiation Oncology)

The Economic Consequences of Artificial Intelligence

• Nick Bloom (Economics)

• Matt Gentzkow (Economics)

• Pete Klenow (Economics)

• Caroline Hoxby (H&S, Hoover Institution, SIEPR)

• Tim Bresnahan (Economics)

• Michael Webb (Economics)

Uncovering gender inequalities in East Africa: Using AI to gain insights from media data

• Gary Darmstadt (Pediatrics-Neonatology)

• James Zou (Biomedical Data Science)

• Londa Schiebinger (History)

• Ann Weber (Pediatrics)

• Valerie Meausoone (Population Health Sciences)

Developing Artificial Intelligence Tools for Dynamic Cancer Treatment Strategies

• Daniel Rubin (Radiology, Biomedical Data Science)

• Michael Gensheimer (Radiation Oncology)

• Susan Athey (Business)

• Ross Shachter (Management Science and Engineering)

• Jiaming Zeng (Management Science and Engineering)

Want out? Removing Individuals’ Data from Machine Learning Models

• James Zou (Biomedical Data Science)

• Greg Valiant (Computer Science)

• Amy Motomura (Law)

2018 HAI Seed Grant Recipients

Adversarial Examples for Humans?

• Gregory Valiant

• Noah Goodman

Automated Moderation of Small Group Deliberation

• Ashish Goel

• James Fishkin

“Always On” Genetic Patient Diagnosis

• Gill Bejerano

• Jon Bernstein

Correcting Gender and Ethnic Biases in AI Algorithms

• James Zou

• Londa Schiebinger

• Serena Yeung

• Carlos Bustamante

Dynamic Artificial Intelligence-Therapy for Autism on Google Glass

• Dennis Wall

• Tom Robinson

• Terry Winograd

Enabling Natural-Language Interactions in Educational Software

• Alex Kolchinski

• Sherry Ruan

• Dan Schwartz

• Emma Brunskill

Fast, Multiphase Human-in-the-loop Optimization of Exoskeleton Assistance

• Steven Collins

• Emma Brunskill

Free Exploration in Human-Centered AI Systems

• Mohsen Bayati

• Ramesh Johari

Gender Bias in Conversations with Chatbots

• Katie Roehrick

• Jeff Hancock

• Byron Reeves

• Londa Schiebinger

• James Zou

• Garrick Fernandez

• Debnil Sur

Harnessing AI to Answer Questions about Diversity and Creativity

• Dan McFarland

• Londa Schiebinger

• James Zou

The Impact of Artificial Intelligence on Perceptions of Humanhood

• Benoît Monin

• Erik Santoro

The Impact on Society of Autonomous Mobile Robots: A Pilot Study

• Marco Pavone

• Mark Duggan

• David Grusky

Improving Refugee Integration Through Data-Driven Algorithmic Assignment

• Jens Hainmueller

• Kirk Bansak

• Andrea Dillon

• Jeremy Ferwerda

• Dominik Hangartner

• Duncan Lawrence

• Jeremy Weinstein

Learning Behavior Change Interventions At Scale

• Michael Bernstein

• James Landay

Learning Decision Rules with Complex, Observational Data

• Xinkun Nie

• Stefan Wager

Learning Haptic Feedback for Motion Guidance

• Julie Walker

• Andrea Zanette

• Mykel Kochenderfer

• Allison Okamura

Mining the Downstream Effects of Artificial Intelligence on How Clinicians Make Decisions

• Ron Li

• Jason Ku Wang

• Lance Downing

• Lisa Shieh

• Christopher Sharp

• Jonathan Chen

New Moral Economy in an Age of Artificial Intelligence

• Margaret Levi

• Justice Mariano-Florentino Cuellar

• Roberta Katz

• John Markoff

• Jane Shaw

Novel Approach to Map Seasonal Changes in Infection Risk for Schistosomiasis: a Multi-Scale Integration of Satellite Data and Drone Imagery by Using Artificial Intelligence

• Giulio De Leo

• Susanne Sokolow

• Eric Lambin

• Zac YC Liu

• Chris Re

• I. Jones

• R. Grewelle

• A. Ratner

• A. Lund

Planning for Multi-Modal Human-Robot Communication

• Yuhang Che

• Cara Nunez

• Allison Okamura

• Dorsa Sadigh

Scaling Collection of Labeled Data for Creating AI Systems through Observational Learning

• Daniel Rubin

• Chris Re

• Jared Dunnmon

• Alex Ratner

• Darvin Yi

Smart Learning Healthcare System for Human Behavior Change

• Michelle Guo

• Fei-Fei Li

• Arnold Milstein

Using AI to Facilitate Citizen Participation in Democratic Policy Deliberations

• Deger Turan

• Frank Fukuyama

• Jerry Kaplan

• Larry Diamond

• Eileen Donahoe

• Chris Potts

Using Computer Vision to Measure Neighborhood Variables Affecting Health

• Jackelyn Huang

• Nikhil Naik

Using Deep Learning for Imaging Alzheimer’s Disease with Simultaneous Ultra-low-dose PET/MRI

• Greg Zaharchuck

• Bill Dally

• John Pauly

• Elizabeth Mormino