Let’s play a little game. Imagine that you’re a computer scientist. Your company wants you to design a search engine that will show users a bunch of pictures corresponding to their keywords — something akin to Google Images.
On a technical level, that’s a piece of cake. You’re a great computer scientist, and this is basic stuff! But say you live in a world where 90 percent of CEOs are male. (Sort of like our world.) Should you design your search engine so that it accurately mirrors that reality, yielding images of man after man after man when a user types in “CEO”? Or, since that risks reinforcing gender stereotypes that help keep women out of the C-suite, should you create a search engine that deliberately shows a more balanced mix, even if it’s not a mix that reflects reality as it is today?
This is the type of quandary that bedevils the artificial intelligence community, and increasingly the rest of us — and tackling it will be a lot tougher than just designing a better search engine.
Computer scientists are used to thinking about “bias” in terms of its statistical meaning: A program for making predictions is biased if it’s consistently wrong in one direction or another. (For example, if a weather app always overestimates the probability of rain, its predictions are statistically biased.) That’s very clear, but it’s also very different from the way most people colloquially use the word “bias” — which is more like “prejudiced against a certain group or characteristic.”
The problem is that if there’s a predictable difference between two groups on average, then these two definitions will be at odds. If you design your search engine to make statistically unbiased predictions about the gender breakdown among CEOs, then it will necessarily be biased in the second sense of the word. And if you design it not to have its predictions correlate with gender, it will necessarily be biased in the statistical sense.
So, what should you do? How would you resolve the trade-off? Hold this question in your mind, because we’ll come back to it later.
While you’re chewing on that, consider the fact that just as there’s no one definition of bias, there is no one definition of fairness. Fairness can have many different meanings — at least 21 different ones, by one computer scientist’s count — and those meanings are sometimes in tension with each other.
“We’re currently in a crisis period, where we lack the ethical capacity to solve this problem,” said John Basl, a Northeastern University philosopher who specializes in emerging technologies.
So what do big players in the tech space mean, really, when they say they care about making AI that’s fair and unbiased? Major organizations like Google, Microsoft, even the Department of Defense periodically release value statements signaling their commitment to these goals. But they tend to elide a fundamental reality: Even AI developers with the best intentions may face inherent trade-offs, where maximizing one type of fairness necessarily means sacrificing another.
The public can’t afford to ignore that conundrum. It’s a trap door beneath the technologies that are shaping our everyday lives, from lending algorithms to facial recognition. And there’s currently a policy vacuum when it comes to how companies should handle issues around fairness and bias.
“There are industries that are held accountable,” such as the pharmaceutical industry, said Timnit Gebru, a leading AI ethics researcher who was reportedly pushed out of Google in 2020 and who has since started a new institute for AI research. “Before you go to market, you have to prove to us that you don’t do X, Y, Z. There’s no such thing for these [tech] companies. So they can just put it out there.”
That makes it all the more important to understand — and potentially regulate — the algorithms that affect our lives.
