AI & Music

Die Seite ist nur in englischer Sprache verfügbar.

The connection between mathematics, mechanics and music has a long tradition in various cultural histories. From ancient Babylon and China, via Egypt and India, knowledge about the entanglement of these fields also reached today’s Europe and has shaped theories and practices in the arts, sciences and techniques of music production and reception for over 2500 years. The Ars Electronica Center has dedicated an exhibition to this topic since 2019. Under the title AI & Music, Ars Electronica traces 1000 years of cultural and technical history of music automatons, machines and systems for composition and reproduction by means of a wide variety of exhibits:

From the writing al-Āla allatī tuzammir bi-nafsihā of Banū Mūsā ibn Shākir, the description of the first programmable music automaton from Baghdad in the 9th century, to the Bösendorfer 290 Imperial CEUS computer grand piano, which is equipped with one of the most advanced and precise integrated recording and reproduction systems available today.

The grand piano was linked by the Ars Electronica Futurelab to four applications from current research into artificial intelligence in music. Performance RNN  from the Google project Magenta is based on a recurrent neural network that independently generates polyphonic music using a long/short term memory. You can still hear rather clumsy improvisations. Piano Genie, also by Magenta, lets visitors play the 88-key piano via an 8-key controller. The interface simplifies and supports making music for all users. The Deep Neural Network MuseNet from the research organization Open AI can imitate and combine different styles. As the GPT-2 language model, also presented at the Ars Electronica Center, MuseNet works with a uniquely large data set. From this pool of musical pieces, with their recurring patterns in harmony, rhythm and tone sequence, the statistical model learned structures from which the AI can now generate something new.

The possibilities of true-to-life recording via the Bösendorfer Imperial 290 CEUS are demonstrated by a recording of the interpretation of Ravel’s Ma Mère l’Oye by Maki Namekawa and Dennis Russell Davies. Highly sensitive sensors register in detail all tone-generating movements, from the speed of the key movements to the position of the pedal, in order to be able to reproduce the intonation exactly. The various pieces and improvisations are accompanied by visualizations by Cori O’Lans who, together with the LA Philharmonic, also developed the orchestral version of the Ravel piece for the Abu Dhabi Festival 2016. For the piano interpretation as well as for the piano phase improvisations, a new set of visualizations has now been created in co-production with the Ars Electronica Futurelab.

The exhibition focused not only on the historical connection between music and the mathematics or mechanics, but also on the tension that for many visitors arises between the emotionality and deeply sensual experience of this medium and mathematics and mechanics, which are understood as purely rational and non-sensual. The social acceptance of AI technologies in the field of music seems to be much more challenging – and seems to cause more irritation – than the automated production of texts, statements and decisions. The programs exhibited by the Ars Electronica Futurelab are intended to give visitors the opportunity to hear for themselves how these technologies are applied in music and what results they create, as well as to develop a basic knowledge on which to base their own assessment and opinions.


Ars Electronica Futurelab: Ali Nikrang, Arno Deutschbauer
Gerfried Stocker
PARTNER: Ars Electronica Center

Related Projects

Take a look at some of our other projects

From our never-ending list of ideas and concepts we have compiled a selection of works related to the topics addressed in this project. An overview of all our productions, cooperations and projects can be found in our continuously growing project archive.