As AI accelerates, can Europe keep up?

Artificial intelligence is developing in leaps and bounds as new systems harness the capabilities of supercomputers and dedicated semiconductors.  But the ongoing acceleration in AI performance and utility raises a number of challenges for policymakers grappling with how to ensure their countries remain competitive and how to regulate this dynamic and far-reaching technology.

Those were two of the messages delivered by the experts participating in a Science|Business Data Rules workshop on the Impact of Advanced Computing on AI. In particular, the meeting explored the ramifications of the recent step change in the size of AI models – the neural networks that learn how to perform specific tasks by detecting patterns in training data.

“In the last five years, the largest scale models have increased 300,000 fold in size,” Andreas Ebert, national and industry technology officer of Microsoft told the workshop. “Currently AI models are doubling every three to four months. And even this speed is accelerating.”

But these advances come at a price. Ebert highlighted how the cost of a single training run can now be as much as €50 million. Microsoft and other major tech companies are increasingly investing hundreds of millions of dollars in building the supercomputers required to run massive AI models. At the same time, tech companies are designing semiconductors specifically to support AI, while also using AI to design new hardware.

As HPC (high performance computing) and AI advance in tandem, a virtuous circle is developing, noted Anne Elster, professor of computer science and founder of the Norwegian University of Science and Technology’s HPC-Lab. “You have HPC for AI and you now have AI for HPC. They now impact each other.”

Many neural networks are now so capable that they are referred to as general purpose models in the sense that they can be used to build a very wide range of applications from speech transcription to facial recognition. As these applications vary from low risk to high risk, general purpose systems are difficult to regulate.  

The current draft of the EU’s AI Act may be starting from the wrong point because it fails to take into account how AI models are developed and how they are integrated into systems, and what kind of decisions will be made when they are deployed. Highlighting the importance of the allocation of responsibilities, Cornelia Kutterer, senior director, EU government affairs, AI, privacy and digital policies at Microsoft, noted that the user of the AI system, who is closest to the high-risk scenario in which the system is deployed, makes important decisions that can have an impact on the system’s performance. This is not appropriately taken into account in the draft of the AI Act, she contended.

Just as the user must be made aware of the capabilities and limitations of the AI system they deploy, the AI system provider will need to ensure that software components providers, pre-trained AI models or general purpose AI deliver the relevant information to them, Kutterer added, noting that the AI Act instructs these providers to collaborate, and this will ultimately be reinforced by contracts in the AI stack. The AI Act should continue to focus on AI systems used in high-risk scenarios, she stressed.

The challenge for regulators is that foundational models, some of which are open source, could be applied in multiple different ways for both good and nefarious purposes. While they can be adapted to write software code, predict molecular sequences and perform other valuable tasks, there is also the risk that they are used to create fake videos or images and propagate extremist content. There is also the danger that any biases in the data used to train general purpose models will be carried through to many different AI applications.  

As these general purpose models are highly malleable components, rather than end products, the line between the provider and the user can become very blurred.