The Problem With Biased AIs (and How To Make AI Better)
AI has the potential to deliver enormous business value for organizations, and its adoption has been sped up by the data-related challenges of the pandemic. Forrester estimates that almost 100% of organizations will be using AI by 2025, and the artificial intelligence software market will reach $37 billion by the same year.
But there is growing concern around AI bias — situations where AI makes decisions that are systematically unfair to particular groups of people. Researchers have found that AI bias has the potential to cause real harm.
I recently had the chance to speak with Ted Kwartler, VP of Trusted AI at DataRobot, to get his thoughts on how AI bias occurs and what companies can do to make sure their models are fair.
AI bias occurs because human beings choose the data that algorithms use, and also decide how the results of those algorithms will be applied. Without extensive testing and diverse teams, it is easy for unconscious biases to enter machine learning models. Then AI systems automate and perpetuate those biased models.
For example, a US Department of Commerce study found that facial recognition AI often misidentifies people of color. If law enforcement uses facial recognition tools, this bias could lead to wrongful arrests of people of color.
Several mortgage algorithms in financial services companies have also consistently charged Latino and Black borrowers higher interest rates, according to a study by UC Berkeley.
Kwartler says the business impact of biased AI can be substantial, particularly in regulated industries. Any missteps can result in fines, or could risk a company’s reputation. Companies that need to attract customers must find ways to put AI models into production in a thoughtful way, as well as test their programs to identify potential bias.
Kwartler says “good AI” is a multidimensional effort across four distinct personas:
● AI Innovators: Leaders or executives who understand the business and realize that machine learning can help solve problems for their organization
● AI Creators: The machine learning engineers and data scientists who build the models
● AI Implementers: Team members who fit AI into existing tech stacks and put it into production
● AI Consumers: The people who use and monitor AI, including legal and compliance teams who handle risk management
“When we work with clients,” Kwartler says, “we try to identify those personas at the company and articulate risks to each one of those personas a little bit differently, so they can earn trust.”
Kwartler also talks about why “humble AI” is critical. AI models must demonstrate humility when making predictions, so they don’t drift into the biased territory.
