Technology Economist Susan Athey Adds DOJ Role to Her Multidimensional Career

After winning awards for technical advances in economic theory and industrial organization, serving as chief economist at Microsoft, conducting original research that combines machine learning with econometric modeling, pioneering the new field of tech economics, helping to found and foster Stanford HAI, and launching the Golub Capital Social Impact Lab at Stanford, Susan Athey will now try on a new hat as chief economist at the antitrust division of the U.S. Department of Justice (DOJ).

The wide-ranging nature of her work and career has garnered her respect across multiple academic fields. “Susan is a force of nature. She moves from machine learning to business strategy to technology policy to social impact, producing deep ideas at every turn,” says Jonathan Levin, the Philip H. Knight Professor and Dean at Stanford Graduate School of Business, where Athey is the endowed Economics of Technology Professor.

Although Athey will continue her GSB appointment in a part-time capacity as she steps into her new government role, the shift in her focus offers an opportunity to reflect on the significant impact she has had throughout her career and while at Stanford. 

A Role Model for HAI

Few researchers better exemplify the multidisciplinary mindset fostered by Stanford HAI than Athey. Her propensity for immersing herself in diverse fields of study goes back to her undergraduate days at Duke University, where she graduated in 1991 with a triple-major in economics, mathematics, and computer science.

After earning her Ph.D at Stanford GSB, she went on to become a professor of economics and business at MIT, Harvard University, and then at Stanford starting in 2013, but even within the field of economics, Athey’s interests have been diverse: In 2007, she won the prestigious John Bates Clark Medal for her contributions to multiple subfields, including industrial organization, microeconomic theory, and econometrics.

But it was during a leave from academia to serve as Microsoft’s chief economist from 2008 to 2013 that Athey made a surprising connection between her passion for economics and the tools of AI and machine learning. 

Athey already knew that digitization and tech platforms were going to play a significant role in the economy, and that search engines were poised to have an outsized importance. She also knew that the research community was just beginning to tackle questions about how to design digital markets and what healthy competition looked like in those markets, and she was excited to help develop that research.

But once she started working at Microsoft, Athey also discovered something she wasn’t expecting: the potential for machine learning to address economic problems. The creators of the Bing search engine were conducting experiments in a manner that economists only dreamed about. They were simultaneously running thousands of randomized A/B tests – asking large numbers of “what if” questions to better understand such things as which search results should rise to the top and how to run auctions for setting advertising prices on a search page. By comparison, she says, economists would typically run one experiment in a year. 

“Microsoft was using an artificial intelligence system composed of hundreds of algorithms that were all working together to make a search results page,” she says. “That was something new.”

In the field of economics until then, data mining and machine learning had been pejorative terms for a less advanced form of statistics. “They were seen as a mechanical exercise to separate cats from dogs,” she says. But at Microsoft, Athey saw an opportunity to combine the computational advances from predictive machine learning with statistical theory so that researchers could better understand causal effects not only in business applications like the search engine, but also in social science and economics. It was an epiphany that launched her in a new research direction and helped define her as one of the early tech economists.

Machine Learning and Causal Effects

Coming out of her experience at Microsoft, Athey realized that the insights from predictive algorithms could be harnessed in new ways by combining them with recent developments in econometrics and statistics. For example, machine learning algorithms could be tailored to answer cause-and-effect questions in economics, such as what will happen if we change the minimum wage? Expand immigration policy? Raise prices? Allow two firms to merge? “Predictive machine learning can’t solve these questions alone, but it can help,” she says.

For example, Athey has used machine learning to look at the impact on consumers of personalized pricing, a form of price discrimination that involves charging different prices to consumers according to their willingness to pay. Traditional economics methods would give aggregate solutions to that problem, she says. They would study perhaps one product category at a time, considering demand for, say, different brands of yogurt or towels. By applying machine learning methods to consumers’ historical purchase data, Athey’s research group can estimate personalized consumer preferences across multiple products at the same time.

Building these predictive models of consumer choice in turn allows researchers to ask even bigger questions about such things as what happens to prices if you apply a tariff, or if generics arrive on the market. “As an input to answering these questions, we want to understand how consumers make choices,” Athey says. And machine learning delivers that input in a way that allows researchers to do this work more efficiently, at a larger scale, and in a more personalized way. “If you’re assuming everybody’s the same, that gives different answers than if you assume that people have different preferences,” she says.

A Tech Economics Pioneer

Athey’s chief economist role at Microsoft ended in 2013, but her tenure there defined her as one of the first people to be considered a “tech economist.” It’s a field she has since helped establish as an independent discipline by convening early conferences in the field and mentoring numerous students along that career path.

“Now tech economists hold an annual conference that draws about 800 participants,” she says. “And we have a specialized job market because being a tech economist is a distinct profession that people can pursue.”

Athey has also written about what it means to be a tech economist. “It’s partly a career, but it’s also a combination of different fields of study,” she says. Tech economists study the impact of digitization on the economy, which entails thinking about market design, privacy, data security, fairness, competition policy, and more, she says. “They also help create and analyze business models and competitive strategy, and they connect the models to data to guide decisions.” 

Advancing AI for Good

At Microsoft, in addition to taking an unexpected deep dive into machine learning and AI, Athey witnessed firsthand the challenges created by these technologies – ethical and legal issues, First Amendment problems, fairness and bias, privacy and copyright, and the prevalence of unforeseen consequences as people manipulated or gamed the system in response to market shifts or new rules.

Because of these observations, Athey developed a desire to impact the ways that machine learning and AI would play out in the world. When she returned to academia full time, her initial steps in that direction included helping to plan the launch of HAI and then becoming one of HAI’s founding associate directors. “Stanford HAI was really created to address these problems,” she says. “We want to make AI beneficial for humans, and we want to avoid all of these unintended consequences.”

Athey also wanted to translate the successful uses of AI from the for-profit sector into the social impact sector. This urge led her to launch the Golub Capital Social Impact Lab at Stanford. “We’re bringing the tech toolkit to social impact applications,” she says. So, for example, the Social Impact Lab has conducted case studies of digital education technology to enhance students’ learning; developed and implemented approaches to targeting educational messaging to maximize engagement of farmers; developed and evaluated digital tablet applications that guide nurses through counseling patients; and developed methods to prioritize candidates for clinical trials of COVID-19 medications.

Connecting the Dots to the DOJ 

Applying machine learning to interesting social problems at the Golub Capital Social Impact Lab is a bottoms-up approach to making change, Athey says. By contrast, in her new job as chief economist of the antitrust division at the DOJ, she will be trying her hand at addressing the problems of the digital economy from the top down. “Government laws and policies affect everything from how competition works to what mergers go through, to what investments people make,” Athey says.

In moving to the DOJ, Athey hopes to continue many of HAI’s efforts to help governments adapt to an era of rapidly changing technology, particularly around the use of data in industry and in government. “Because technology such as artificial intelligence moves so quickly, it’s hard for the government to keep up,” she says. “We have to figure out how all branches of government are going to be prepared to guide us through a different age.”

It’s a great time for Athey to try her hand at government work, says Levin. “At a moment when technology is ascendant and promoting competition is essential, I can’t think of anyone I’d rather have at the DOJ than Susan.”

Athey will remain a professor at Stanford Graduate School of Business and a senior fellow at the Stanford Institute for Economic Policy Research. She will step down from her Stanford HAI associate director role although she will continue to be an affiliated faculty member.

Stanford HAI’s mission is to advance AI research, education, policy and practice to improve the human condition. Learn more.