Your life in AI’s hands: The battle to understand deep learning
As society enters an era where AI will take life or death decisions—spotting whether moles are cancerous and driving us to work—trusting these machines will become ever more important.
The difficulty is that it’s almost impossible for us to understand the inner workings of many modern AI systems that perform human-like tasks, such as recognizing real-life objects or understanding speech.
The models produced by the deep-learning systems that have powered recent AI breakthroughs are largely opaque, functioning as black boxes that spit out a result but whose operation remain mysterious. This inscrutability stems from the complexity of the large neural networks that underpin deep-learning systems. These brain-inspired networks are interconnected layers of algorithms that feed data into each other and can be trained to carry out specific tasks. The way these systems represent what they have learned is spread across these sprawling and densely connected networks, and dispersed in such a way that their workings are very tricky to make sense of.
Technology giants such as Google, Facebook, Microsoft and Amazon have laid out a vision of the future where AI agents will help people in their daily lives, both at work and at home: organizing our day, driving our cars, delivering our goods.
But for that future to be realized, machine learning models will need to be open to scrutiny, says Dr Tolga Kurtoglu, CEO of PARC, the pioneering Silicon Valley research facility renowned for work in the late 1970s that led to the creation of the mouse and graphical user interface.
“There is a huge need in being able to meaningfully explain why a particular AI algorithm came to the conclusion it did,” he said, particularly as AI increasingly interacts with consumers.
“That will have a profound impact on how we think about human-computer interaction in the future.”
Systems will need to be able to articulate their assumptions, which paths they explored, what they ruled out and why, and how they arrived at their conclusion, according to Kurtoglu.
“It’s the first step towards establishing a trusted relationship between human agents and AI agents,” he said, adding that collaboration between humans and machines could prove highly effective in solving problems.
Greater insight into an AI‘s workings would also help identify where faulty assumptions originated. Machine learning models are only as good as the training data used to create them, and inherent biases in that data will be reflected in the conclusions these models reach.
