Demographic Stereotypes in Text-to-Image Generation

Key Takeaways

• Text-to-image generative AI usage is growing, but the outputs of state-of-the-art models perpetuate and even exacerbate demographic stereotypes that can lead to increased hostility, discrimination, and violence toward individuals and communities.

• Stable Diffusion generates images that encode substantial biases and stereotypes in response to ordinary text prompts that mention traits, descriptions, occupations, or objects—whether or not the prompts include explicit references to demographic characteristics or identities. These stereotypes persist despite mitigation strategies.

• DALL-E similarly demonstrates substantial biases, often in a less straightforward way, despite OpenAI’s claims that it has implemented guardrails.

• Technical fixes are insufficient to address the harms perpetuated by these systems. Policymakers need to understand how these biases translate into real-world harm and need to support holistic, comprehensive research approaches that meld technological evaluations with nuanced understandings of social and power dynamics.

Executive Summary

Text-to-image generative artificial intelligence (AI) systems such as Stable Diffusion and DALL-E that convert text descriptions provided by the user into synthetic images are exploding in popularity. However, users are often unaware that these models are trained on massive datasets of images that are primarily in English and often contain stereotyping, toxic, and pornographic content. As millions of images are generated each day using these AI systems, concerns around bias and stereotyping should be front and center in discussions.

In a new paper, “Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale,” we show that major text-to-image AI models encode a wide range of dangerous biases about different communities. Past research has demonstrated these biases in previous language and vision models, with recent research starting to explore these issues in relation to image generation. This paper aims to highlight the depth and breadth of biases in recently popularized text-to-image AI models, namely Stable Diffusion and DALL-E. We test a variety of ordinary text prompts and find that the resulting images perpetuate substantial biases and stereotypes—whether or not the prompts contain explicit references to demographic attributes.

Our research underscores the urgent need for policymakers to address the harms resulting from the mass dissemination of stereotypes through major text-to-image AI models.

Finally, our third study aimed to identify previously unknown complex undercompensated groups defined by multiple attributes. After considering age, documented sex, and 12 chronic health indicators, we found many groups with multiple chronic conditions that were undercompensated by at least $10,000 and up to $29,600. For instance, enrollees with asthma, heart disease, and mental health and substance use disorders were undercompensated by about $12,000.

Policy Discussion

The patients at the center of our studies are marginalized in the healthcare system and many problems in the system persist. Health plan enrollees with multiple chronic conditions already face challenges in maintaining access to care. Patients with undercompensated health conditions also represent a considerable portion of the U.S. population— approximately 20 percent of the country’s population have a mental health or substance use disorder. 

Policymakers should use their convening power to bring together industry and patient advocacy groups to parse out the consequences of new fairness metrics.

Subscribe to Policy Program Updates

Our second study found that a combination of removing drug and certain health condition variables, using 1 percent of funds for reinsurance, and introducing fairness constraints on the loss function for four undercompensated chronic illnesses greatly improved overall fit and fairness. This approach also improved group fit for most undercompensated groups not included in the loss function.