Making Sense of Sound: What Does Machine Learning Mean for Music?
AI has proven to have a considerable impact on some major industries. While autonomous cars and virtual assistants are slowly becoming a reality, the creative industry has been experimenting with AI for several years already. Does it have meaningful implications and if so, what will it bring in the future?
It’s universally agreed that the first computer-assisted music score dates back to 1957 when composers Lejaren Hiller and Leonard Isaacson unveiled Illiac Suite for string quartet. Utilizing the interconnection between mathematics and music, Hiller was able to program the computer to come up with a stunning four-piece musical score.
One of the most notable AI-assisted music projects happened two years ago. Singer-songwriter Taryn Sothern decided to make AI a centerpiece of her music production workflow, releasing the album I Am AI. For the first time in history, all of the composition and production on the album was done by collaborative efforts of four AI programs, machine learning consultants, and a musician, those being Amper Music, IBM Watson Beat, Google Magenta, AIVA, and Taryn herself.
It’s important to note that AI didn’t produce the songs from start to finish. It just generated ideas based on set parameters. Afterward, all the stems had to be adjusted, structured, and mixed by a human. However, we are definitely not far from the point when a machine can write a song entirely on its own.
In 2018, French music producer Skygge released a multi-genre album composed in collaboration with the AI system Flow Machines. The album was regarded as the first ‘good’ AI album by BBC Culture. The co-producer of the album, Michael Lovett, perfectly sums up the experience of writing music with AI: “It’s a bit like having somebody playing a piano in the corner of your studio. They’re kind of playing stuff and most of it is rubbish, but at some point you’re like, ‘Oh, what’s that? That’s interesting.’ And it’ll become the jumping-off point for a song.”
And what about the lyrics? Researchers at the University of Antwerp and the Meertens Institute created a rap lyrics generator called Deep Flow. By feeding the algorithm with an enormous number of hip-hop lyrics, the machine has learned to ‘spit’ arguably believable lyrics of its own. In case you want to prove the creators wrong, you can play the game on the Deep Flow website that challenges the players to tell machine-generated lyrics from the ones from actual songs.
Right now, computer-generated music is mostly found in functional and background compositions. For example, amateur filmmakers and YouTubers often face the challenge of finding original music for their works. Not only is it often hard to find the right artist, but content creators also need to consider the time it takes an artist to produce music and the costs associated with it. Amper Music can generate an entire song based on certain parameters like key, tempo, style, and mood in a matter of minutes. Such technology may significantly lower the entry barriers to the creative industries.
While music is a creative outlet, technology has always been a huge part of the industry. Nowadays, even if a certain song features recordings of real instruments, it requires a very specific skillset, extended knowledge, and sufficient experience to process these recordings properly.
Such work is usually done with the help of digital audio workstations (DAWs), hardware equipment and software plugins. Currently, the most advanced audio software developers are looking for ways to adapt AI in their plug-ins. Let’s look at some notable examples.
US-based company iZotope heavily invested in machine learning applications in recent years. For example, their intelligent channel strip plugin Neutron uses deep learning to identify what kind of instrument is being played, and based on its assumption recommends the user which EQ or compression settings to apply.
