Three Coming Shifts In AI

Nearly every new day brings exciting news in the field of artificial intelligence. But what larger directional trends do these news items drive? Beyond the announcements and the hype, is AI really evolving?

In this article, I’d like to focus not on far-off, vague hopes and wishes about AI, but instead on a few concrete developments that lie in the not-so-distant future. The trends outlined below are already beginning to materialize in the form of real-world research and applications. These areas of work represent themes that I believe will be recorded as meaningful breakthroughs in a future timeline of key AI developments.

1.      The Cost of Training Machine Learning Systems Will Be Drastically Lowered

The AI community has long understood that our most successful methods, such as deep learning, are mathematically and computationally complex. Using these methods with current techniques involves the consumption of significant amounts of processing resources and a high degree of cost. This, in turn, limits where and how liberally these techniques can be applied. But less expensive training techniques are coming to the rescue. It is important to understand that the cost savings and speed increases ordinarily driven by Moore’s Law are entirely separate from the types of algorithmic efficiencies discussed below. While Moore’s Law promises to double transistor counts—and hence processing power—every two to three years, algorithmic breakthroughs can often lead to instant efficiencies that are orders of magnitude more significant. What could some of these breakthroughs look like?

One promising area is the development of lightweight neural networks, which are exactly what they sound like: smaller, quicker to train networks that can achieve almost the same accuracy as their much larger cousins, but at a fraction of the size and training cost. Research led by SparkCognition’s Chief Scientist, Dr. Bruce Porter, revealed that in cybersecurity applications, lightweight neural networks could match conventional deep networks despite using a mere tenth of the computational capacity required for those larger structures.

Another emerging technique, sparse learning, holds the potential of speeding up the training of deep networks by 3.5 to 12 times. Separately, researchers at North Carolina State University have shown that by taking advantage of the similarities in sub-segments of images in a training dataset, they can potentially reduce the amount of time and compute power required for network training by more than 60%… all without sacrificing accuracy.

This is by no means a complete list of the considerable volume of work dedicated to reducing the computational burden of training deep networks. But even this small glimpse shows that the near future will likely bring far more efficient methods that will allow us to rival the decision-making power of large, complex networks without expending the same computational resources or compromising accuracy.

So, what are the implications of these coming advancements? Here are at least a few:

1. With a significantly reduced computational burden, networks will become easier to deploy at the edge, in embedded systems and in environments where access to cloud resources is not possible. This will allow machine learning and AI to proliferate into (smaller and smaller) objects and “things” in the real world.

2. Since AI running at the edge means that the in-field systems it enables can be self-sufficient in terms of their intelligence, such systems will also be capable of supporting disconnected operations. Think of naval ships operating in denied environments, aircraft operating in areas without satellite coverage, or underwater systems that cannot communicate using conventional high-data rate technologies.