How New Hardware Can Drastically Reduce the Power Consumption of Artificial Intelligence
To make predictions, artificial intelligence (AI) relies on the processing of large amounts of data and that takes a lot of energy. Imec develops solutions to drastically reduce that energy consumption. A new chip, in which these calculations are performed directly in the memory by means of analogue technology, is a major breakthrough in this field.
Our planet’s warming up. Addressing that problem requires a climate transition. This transition is the driving force behind the European Commission’s work programme for 2020, entitled ‘Towards a fair, climate-neutral and digital Europe’. In addition to an ecological transition, we are also facing a digital transition. The digital transformation is therefore the second spearhead of the EU programme, with a particular emphasis on artificial intelligence (AI). When the European Commission states that ‘technologies such as AI will be a crucial factor in achieving the objectives of the Green Deal’, it links both spearheads. The idea behind it: using AI as a weapon in the fight against global warming.
The cost of AI to the planet
Artificial intelligence has a lot of potential, for example, to make more accurate climate forecasts, to detect energy loss or to help decarbonize different industries. On the other hand, the technology itself comes at an ecological cost. After all, it relies on processing huge amounts of data. This already accounts for a substantial part of global electricity production. Moreover, the number of calculations is increasing exponentially. Although these calculations are typically carried out in data centers that have already become much more energy efficient in recent years and partly use renewable energy, the impact on our planet is considerable.
For example, last year researchers from the University of Massachusetts calculated the emissions associated with training models in natural language processing (mainly used to improve translation machines). They did this by briefly testing different algorithms on a single GPU, extrapolating the power consumption with the reported training duration and converting that into CO2 emissions based on the average energy mix at different cloud providers. Conclusion: fine-tuning the most energy-consuming algorithm releases up to 284 tonnes of CO2. This is equivalent to the CO2 emissions of five cars during their entire lifespan, including construction.
A growing awareness is therefore emerging among AI researchers to take more account of the ecological cost of the algorithms they are developing. Recently, researchers have been able to enter their total calculation time and the hardware and cloud service used for this into a Machine Learning Emissions Calculator, after which the estimated CO2 emissions are included in their paper.
In essence, you can reduce that ecological footprint in two different ways: via the software and via the hardware. You can try to develop more efficient algorithms that minimize the number of calculations. A typical example is the technique of network pruning. In doing so, you ‘prune out’ all the parts that have little importance for the end result. What then remains is a neural network that has the same functionality, but is smaller, faster and more energy efficient.
You can also design energy-efficient hardware. Currently, AI calculations are mainly performed on graphical processors (GPUs). These processors are not specially designed for this kind of calculations, but their architecture turned out to be well suited for it. Due to the wide availability of GPUs, neural networks took off. In recent years, processors have also been developed to specifically accelerate AI calculations (such as Google’s Tensor Processing Units – TPUs). These processors can perform more calculations per second than GPUs, while consuming the same amount of energy. Other systems, on the other hand, use FPGAs, which consume less energy but also calculate much less quickly.
