Pianographique – Music and Images by Humans and Machines: AI Reflection #1-3 photo: Tiberio Sorvillo

transart23: Pianographique

Music and Images by Humans and Machines

The program “Pianographique – Music and Images by Humans and Machines” was showcased at the acclaimed multidisciplinary festival transart23 in Bolzano, South Tyrol, on September 21, 2023. It combined two current artistic research projects of Ars Electronica, both dedicated to the creative fusion of instrumental music and digital technologies.

The concert began with AI-composed musical pieces by Ali Nikrang. Nikrang is a Key Researcher and Artist at the Ars Electronica Futurelab. As a composer and computer scientist, he develops systems with which he composes instrumental music in dialogue with artificial intelligence. His AI composition system, Ricercar, is an intuitive interface: the output of the AI composition can be personalized by humans.

By starting the content, you agree that data will be transmitted to vimeo.com.
Data Protection Declaration

Pianographique at transart23 / video: Marina Baldo

Human creativity, AI and a self-playing piano

Composed for a self-playing piano and played as if by magic from a Steinway grand piano, the first piece AI-Reflection #1 demonstrated the current state of AI technology – in the form of a direct reproduction of the AI’s musical output without human accompaniment. This was followed by AI-Reflection #2, a joint performance with three musicians of the MDI Ensemble (as a string trio) and the self-playing piano. AI-Reflection #2 showed another quality of this creative collaboration: music that is created in the harmonious interplay between human and machine, with the self-playing piano clearly taking the musical lead. Finally, the third piece, AI-Reflection #3, was a joint performance of the string quartet and the self-playing piano, with the composition of the AI, as in the second piece, being arranged by human hands for this instrumentation. Here, the self-playing piano (the machine) took on an accompanying role to the musicians.

Nikrang then presented Sounding Letters with real-time visuals by Florian Berger, where the musical theme arose from the idea of representing letters as notes. In 2021, on the occasion of the Futurelab’s 25th anniversary, the initials of the Ars Electronica Futurelab and the Ars Electronica Center were transformed into several pieces of music with the help of Ricercar.

Personality meets generative AI

The second part of the concert was taken over by Maki Namekawa, who brings human-made music to life with her unique virtuosity and sensitivity, supported by real-time visualization by Cori O’Lan. Few pieces could be better suited for this dramatic juxtaposition than Keith Jarrett’s legendary “The Köln Concert,” which she presented in its entirety. This performance premiered at the Ars Electronica Festival in September 2023.

The digital real-time visualizations for both parts of the concert were created in collaboration between human and machine. The visuals, designed by Florian Berger and Cori O’Lan, reacted live to the music, its dynamics, harmonies and timbres, giving rise to the visual worlds of this concert evening.

The images for Maki Namekawas interpretation of “The Köln Concert” were created by Cori O’Lan using generative AI like MidJourney, RunwayML or Stable Diffusion. As a starting point, sessions with ChatGPT generated prompts for image generation, resulting in tens of thousands of pictures that had to be carefully selected. The ambiguity about with whom or with what one is in dialog, with oneself, with a machine, with the enormous amounts of collected data and information and the world views generated from them, on which these AI systems are built, is a dynamic that enriches the artistic process. One always oscillates on the border between the new and repetition, between originality and imitation.

To deepen the topic, Ali Nikrang also discussed “AI, music, and creativity” with Patrick Ohnewein, Head of Unit for Digital Technologies ICT at NOI Techpark, on September 20th as part of transart23.

Credits

Ars Electronica Futurelab: Florian Berger, Ali Nikrang

Piano: Maki Namekawa
AI composition: Ali Nikrang
Real time visualizations: Cori O’Lan, Florian Berger
Strings: mdi ensemble: Corinna Canzian, Elia Leon Mariani (violin), Paolo Fumagalli (viola), Giorgio Casati (cello)