There is often an assumption that technology is neutral, but the reality is far from it. Machine learning algorithms are created by people, who all have biases. They are never fully “objective”; rather they reflect the world view of those who build them. And unless there is concerted intervention, algorithms will continue to reflect and reinforce the prejudices that hold society and business back. We can preempt some of the damage by utilizing ethical AI design principles. We also need to ensure that our algorithms are explainable, auditable, and transparent. Just as we wouldn’t accept humans making major decisions that affect others without any oversight or accountability, we should not accept it from algorithms. We need to start looking at eliminating AI Bias less as merely a “nice thing to do,” and more as an economic and competitive imperative. Business leaders take note: By making our AI systems more fair, we also make our organizations more profitable and productive.
The authority that administers A-Level college entrance exams in the UK, Ofqual, recently found itself mired in scandal. Unable to hold live exams because of Covid-19, it designed and employed an algorithm that based scores partly on the historical performance of the schools students attended. The outcry was immediate, as students who were already disadvantaged found themselves further penalized by artificially deflated scores, their efforts disregarded and their futures thrown into disarray.
This is far from an isolated incident. Even the world’s most sophisticated technology companies have faced similar problems. In 2018, Amazon’s recruiting algorithm was flagged for penalizing applications that contained the word “women’s.” More recently, Apple’s credit card algorithm was so biased against women that founder Steve Wozniak’s wife was given a credit limit 10 times lower than his, despite them sharing all assets and accounts.
Clearly, this is unfair. Just as important, these biases undermine efficiency and productivity. If we see a value in putting the best students in the best schools, the inherent Bias in Ofqual’s algorithm undermines that purpose. Similarly, a biased recruiting algorithm undermines a firm’s ability to attract the best talent, and a biased credit rating algorithm undermines the ability to make smart credit decisions.
Just as the costs of bias are substantial, the benefits of eliminating bias can be just as significant. In fact, one econometric study at Stanford University found that at least “25% of growth in U.S. GDP between 1960 and 2010 can be attributed to greater gender and racial balance in the workplace,” and that the figure could be as high as 40%.
In order to fully unleash the profitable opportunities of AI, we first need to understand where biases come from. There is often an assumption that technology is neutral, but the reality is far from it. Machine learning algorithms are created by people, who all have biases. They are never fully “objective”; rather they reflect the world view of those who build them and the data they’re fed.
Biased human judgments can affect AI systems in two different ways. The first is bias in the data that systems learn from. You can see this play out for yourself: do a Google image search for “professional haircut,” and another for “unprofessional haircut.” “Professional haircut” turns up results that are exclusively white men, while “unprofessional haircut” has much more gender and racial diversity.(This issue was originally surfaced by Twitter users back in 2016.)
Is it really true that only white men have professional haircuts? Of course not. The Google results are based on articles written about professional haircuts. They reflect human editorial decisions to include and prioritize white men. So a supposedly “neutral” search provides a decidedly unneutral outcome. Bias based on historical norms is fairly common.
A second source of bias occurs in the way algorithms are designed.
