Inescapable Logics and Unresolvable Tensions: The Challenges of Tech Industry Ethics Owners

Should Google engineers be able to know what they’re building and for whom — to make decisions about their contributions — such as for Project Dragonfly, an ultimately terminated censored search engine for use in China?

What are the boundaries of personal information used by technology firms, as illuminated by the Cambridge Analytica scandal of 2018?

How can we understand and address racial disparities in algorithms used as part of the criminal justice system?

Such questions highlight “the stakes of Silicon Valley companies and data-driven technologies for all of society,” says Emanuel Moss, a PhD candidate in anthropology at CUNY Graduate Center and researcher with Data and Society and the Pervasive Data Ethics for Computational Research (PERVADE) project. He and Jacob Metcalf, also with Data and Society and PERVADE and a co-founder of consulting firm Ethical Resolve, addressed the challenges of ethics in Silicon Valley as part of a Stanford HAI seminar titled “Owning Ethics: Organizational Responsibility and the Institutionalization of Ethics in Silicon Valley” on May 1, 2020.

The researchers highlighted findings from their study (with danah boyd) published in Social Research; they’ve also written about the ideas in Harvard Business Review and the Data and Society Points Blog.

Broadly, they uncover inescapable logic and unresolvable tensions that tech-firm ethics owners face, while suggesting ways to move industry ethics practices forward.

Who Are the Ethics Owners?

To understand tech-industry ethics, the researchers sought out those dealing directly with ethical issues.

They interviewed people with titles like responsible AI lead, ethical AI practice lead, civil liberties lead, and heads of policy, product, and legal teams at a large number of Silicon Valley tech companies. Some were engineers who were inspired by the ethics and politics of technology, Metcalf says. Others came from social science, law, and humanities fields.

“We call these people ‘ethics owners,’” Moss says. “They own responsibility for ethics practices across an organization.”

Inescapable Logics and Unresolvable Tensions

Several Silicon Valley logics provide challenging roadblocks to these ethics owners, the speakers note.

First, meritocracy “rationalizes inequality by linking the unequal distribution of wealth and power to natural differences in individual ability,” Moss says, “which is pervasive in Silicon Valley.” At times, this logic can dismiss critique, or make ethics an individual decision, removing institutions from blame.

Second, technological solutionism — the idea that technology can solve every problem — also presents a challenge. Even when those in the technology industry acknowledge their products have created social problems, like contributing to income inequality, they too often propose technology solutions. This logic gives the “impression that ethics problems arise from imperfect technical solutions,” Moss says, “when many of the most intractable social impacts of technology arise from products that work exactly as intended.”

A third common logic, “market fundamentalism,” or letting markets decide good and bad ideas, often leads ethics owners to implement practices that do not affect a company’s bottom line. “Ethics initiatives often need to be couched in the language and concepts of the market,” Moss says, and some people he interviewed felt a cynicism that market success trumped ethics.

Due to these challenges in logic, ethics owners must navigate unresolvable tensions, responding to “If we can’t measure it, we can’t fix it” attitudes, for example, or working within a narrowly defined risks/rewards framework where it’s easier for ethics owners to advocate for changes that limit potentially harmful effects of a product than for those that may benefit users, Metcalf says. The task of ethics owners consists of working through, not resolving, these tensions, the scholars say.

The Way Forward

In this context, ethics owners can best influence “by disseminating ideas across the company and across their peer network of other ethics owners,” Moss says. Ethics-related case studies are critical for this. Ethics owners can produce studies from their own domains and commission them from affected populations. “These capture what did and didn’t work in the past and create lines of communication among ethics owners,” Moss continues.

Cases that contain unique, idiosyncratic details specific to subsectors and firm practices reveal key details of successful and unsuccessful projects, elevating their value above that of generic studies.

The researchers noted some ethics owners are already sharing some cases semi-secretly, in data ethics salons or informal gatherings. “These spaces can serve as a model of trust,” Moss says, advocating for “recurring meetings where employees can discuss issues frankly without pre-clearing by the press department.”

It’s also crucial to engage with the concerns of those affected, such as historically disadvantaged groups. “That’s vital for any ethics practice that centers care and dignity over more narrow, quantified framings of risk and liability,” Moss says.

In addition, buy-in from top leadership also influences ethics owners’ impact, as owners work to “translate between broader societal pressure and actual corporate governance practices,” as Metcalf says. Government, for its part, can mandate due diligence practices, such as requiring companies using automated decision making to articulate a mode of bias prevention.

In closing, Moss says, “The job of ethics owners isn't so much to make the company ethical, but to make the ethics of the company work in many different ways.” This leads to a bigger question, the researchers note: Can the tech industry be ethical if it does not call into question Silicon Valley’s underlying logic?

Moss and Metcalf spoke at a recent Stanford HAI seminar. Watch the full discussion.