Why is Artificial Intelligence so Energy Hungry?

Artificial intelligence (AI) systems today are still far from replicating true human intelligence. But, they are certainly getting better at discerning data patterns and mining insights, to a degree better than us. At present, artificial intelligence models can recognize images, converse with people via chatbot, drive autonomous cars and even has won against us in Chess. However, did you know, that the energy and power consumption involved in training and building these models is highly staggering? In other words, training AI is an energy-intensive process with a high carbon footprint.

So, reducing this energy consumption will have positive knock-on effects on the environment. Further, it will also bring other benefits to businesses, such as reducing their carbon footprint and getting closer to carbon-related targets. And before moving ahead with building energy efficient AI or green AI, we must understand why artificial intelligence is so power-hungry?

Let us consider a neural network model. Neural network is a powerful type of machine learning which models itself by mirroring the human brain. Composed of node layers, a neural network attempts to recognize the underlying relationships in a data set by mimicking human brain functions. Each node is connected with another and has an associated weight and threshold. Suppose the output value of a node is higher than the specified threshold value, it implies that the node is activated and ready to relay data to the next layer of the neural network.

The training of a neural network comprises of running forward pass, where input is passed through it and an output is generated after processing the input. Then the backward pass involves updating the weights of neural network using errors received in forward pass, via gradient descent algorithms that need a massive amount of matrix manipulation.

In June 2019, a team of researchers from the University of Massachusetts at Amherst, published a paper on their study, where they assessed the energy consumption required to train four large neural networks. These neural networks are: Transformer, ELMo, BERT, and GPT-2, which they trained on a single GPU for one day each, and measured the energy consumption throughout.

One of these neural networks, viz., BERT (Bidirectional Encoder Representations from Transformers) uses 3.3 billion words from English books and Wikipedia articles. According to an article on The Conversation by Kate Saenko, BERT had to read this vast data set around 40 times in the training stage. To draw a comparative analysis, she mentions that a five-year-old, average child learning to talk might hear 45 million words by this age, which is 3000 times fewer than BERT.

In the study from the University of Massachusetts at Amherst, the researchers found that training BERT once has the carbon footprint of a passenger flying a round trip between New York and San Francisco. The team computed the total power consumption for training each model by multiplying this figure by the total training time reported by each model’s original developers. The carbon footprint was calculated on the basis of average carbon emissions used in power production in the US.