Melody of Crisis/Joy

Gregor Pechmann (AT), Floria Rothkegel (AT), Markus Schedl (AT), Alexander Wallner (AT), Emilia Parada-Cabaleiro (ES), Vasco Fragoso (PT)

| | |

Melody of Crisis/Joy is an interdisciplinary project aiming to benefit from Artificial Intelligence within an artistic setting. In order to promote awareness of how globally occurring crisis events are communicated online, emotions conveyed by spoken statements are identified through sentiment analysis and speech emotion recognition, then sonified into musical melodies. For the transformation, recordings by several musicians expressing positive and negative emotions are used to train two machine learning models able to sonify positive/negative statements into crisis/joy melodies, respectively. When entering the installation, an interactive audiovisual world map where countries can be selected to listen to their corresponding melodies is presented and the visitor is invited to guess the type of melody (Crisis/Joy); subsequently, the real statement is played back with subtitles. Visitors are also invited to interact with intelligent musical sculpture able to sonify their voices in real-time.


Gregor Pechmann (Social Media Manager at the University Communications Department) together with Emilia Parada-Cabaleiro and Markus Schedl (Institute for Computation Perception and Human-Centered AI Group) created (with the support of JKU AI Students Alexander Wallner, Florian Rothkegel, and Vasco Fragoso) an immersive experience of the global sentiment shared in social media towards crisis events.


Johannes Kepler University Linz (Antonia Ebner, Vasco Fragoso, Emilia Parada-Cabaleiro, Gregor Pechmann, Martin Reiter, Florian Rothkegel, Theresa Rubenzucker, Markus Schedl, Alexander Wallner); Musicians from the Anton Bruckner Private University (Ivan Bykov, Johanna Dömötör, Anna Maria Dörfler, Bernhard Hadriga, Matthias Kronsteiner); Land Oberösterreich