Making Local Law Legible: LLM-Assisted Detection of Discrimination

Key Takeaways

• Understanding local law can be overwhelming. Historically, reformers have hired large teams of researchers and lawyers to sift through federal and state codes, but that is particularly infeasible for thousands of cities, counties, and towns across the United States. 

• Stanford HAI and RegLab researchers gathered 9,623 jurisdictions’ local laws (amounting to 3 billion words of code) and developed an LLM-assisted review pipeline to demonstrate the potential of AI for legal reform. Together, these laws govern in jurisdictions that are home to roughly 252 million Americans. 

• We identify patently discriminatory laws. Shockingly, we document dozens of jurisdictions that formally still mandate racial segregation, impose poll taxes, and deny women the right to vote, practices that have long been held unconstitutional. A particularly prevalent category is unwarranted citizenship-based discrimination for occupational licenses. 

• These methods empower lawyers, local governments, and citizens to understand and target reform efforts. LLM-assisted statutory surveys have the potential to update and make the legal code legible again. 

The Problem

Local law matters. Across the United States, local law draws lines determining who can vote, who can hold particular jobs, who can attend which schools, and who can be detained or punished. Historically, local lawmakers have often drawn those lines unjustly across protected categories like race, sex, and religion. Ridding codes of these laws removes enforcement risks and public misunderstanding and clears historical stains.

However, making local law legible takes time. Until recently, reformers combed through codes by hand with small armies of researchers and lawyers. In the 1950s, Pauli Murray compiled over 700 pages of segregation laws — a volume Thurgood Marshall called the “bible” of Brown v. Board of Education. In the 1970s, then-Professor Ruth Bader Ginsburg led an effort to excise sex bias from federal law, searching the law using just 59 gender-related keywords. In the 2000s, law professor Gabriel “Jack” Chin and his team cataloged vestigial Jim Crow provisions in state statutes, triggering repeals in several states — while acknowledging they likely missed unusually phrased laws. As recently as 2019, a Virginia commission compiled 60 years of discriminatory state laws only by mobilizing law students from three universities, the governor’s staff, practicing attorneys, and a sitting judge. The time and labor required for these painstaking searches are significant. No comparable effort has ever been possible across local codes. There are simply too many.

In our paper “Hidden in Plain Text: LLM-Assisted Detection of Discriminatory Local Laws,” we present the first effort to develop AI assistance for surveying discriminatory provisions at scale and systematically across local codes. We gathered the largest dataset of local laws to date — spanning 9,623 jurisdictions and 75% of the U.S. population — and used large language models (LLMs) to flag likely instances of discriminatory laws for subsequent human verification. 

We found plainly discriminatory laws across the United States in nearly every protected category we searched. These laws can be viewed on our paper’s companion website. We present our approach as a promising solution to a search problem that has hindered code cleanup efforts for decades — and as a foundation for further work on statutory surveys that turn on legal, rather than topical, questions. 

The Solution

In previous work from the Stanford RegLab, our colleagues used LLMs for statutory and regulatory research and found outdated, obsolete, and cumbersome legal requirements. Yet those statutory surveys did not assess the potential for AI systems to assist in determining the constitutionality of legal provisions, an inquiry whose answers bear on people’s legal rights and equal civic standing.

Here, we introduce a new LLM-assisted method specifically for identifying unconstitutionally discriminatory laws. The method draws on LLM and human judgment, and focuses on laws that are discriminatory in ways that are identifiable from the text itself when read against current constitutional doctrine. (We focus on “disparate treatment” and not facially neutral laws with discriminatory effects.) This necessarily requires developing an approach that is more attuned to antidiscrimination law and legal reasoning than prior work.

We first assembled a corpus of local laws — covering roughly 75% of the U.S. population, spanning 9,623 jurisdictions, and totaling 3 billion words of code—by scraping and parsing codes from two major codification providers. The dataset contains approximately 9 million law sections.

We then used an LLM to label each legal section based on whether it references a protected category — such as race, religion, or gender — and whether the law imposes differential treatment based on that category. We grouped laws by the type of treatment they impose, allowing reviewers to filter out laws where differentiation may be used for accommodations or remedial purposes. Finally, an LLM scored the remaining provisions on five attributes drawn from antidiscrimination doctrine — attributes that signify the presence of discriminatory treatment without a recognized justification. We used multiple LLMs to assign a priority rating based on LLM agreement to direct human review.