Why we need to look inside the black box of AI
How artificial intelligence engines make decisions is just one of a number of logistic and ethical reasons we need to be investigating systems more closely, says this Stanford University expert
In the late 18 century, Wolfgang von Kempelen amazed the world with his chess playing automaton, the Mechanical Turk, as it defeated worthy opponents including Napoleon Bonaparte and Benjamin Franklin.
Onlookers were amazed at how a device of gears and sprockets had been able to master the complexities of chess and demanded to know what was inside.
As it turned out, the chassis of the Mechanical Turk actually housed a human being – a simple yet highly-effective deceit made possible by the inability of onlookers to take a peek inside.
While the story of the Mechanical Turk is more than 100 years old, it may be a useful parable for modern society’s increasing use of artificial intelligence (AI), and how we are often relying on findings with little understanding of how they are reached.
The opacity of AI systems has significant potential for abuse, particularly through the long-term impact of intentional or even unintentional biases in the data. It is a topic that has also become prominent in thinking of many computer scientists and researchers, and given rise to the concept of Explainable AI.
One of the advocates for the potential of Explainable AI is Stanford University’s Executive Director of Strategic Research Initiatives in Computer Science, Dr Stephen Eglash.
Speaking to CMO ahead of his appearance at the Melbourne Business Analytics Conference on 19 July, Dr Eglash described how we have created AI systems that are effectively ‘black boxes’ whose results and decisions can be upsetting to those they impact when we have no idea of how those results are produced.
“If an AI system is helping a loan officer at a bank determine whether you are eligible for a loan, and it decides to reject your loan application, it’s reasonable to expect an explanation,” Dr Eglash says.
The loan applicant Dr Eglash describes may have no idea how the system came to reject their application – or whether it might have been working from data or an algorithm that was somehow prejudiced against the applicant.
