Melody of Crisis/Joy is an interdisciplinary project aiming to benefit from Artificial Intelligence within an artistic setting. In order to promote awareness of how globally occurring crisis events are communicated online, emotions conveyed by spoken statements are identified through sentiment analysis and speech emotion recognition, then sonified into musical melodies. For the transformation, recordings by several musicians expressing positive and negative emotions are used to train two machine learning models able to sonify positive/negative statements into crisis/joy melodies, respectively. When entering the installation, an interactive audiovisual world map where countries can be selected to listen to their corresponding melodies is presented and the visitor is invited to guess the type of melody (Crisis/Joy); subsequently, the real statement is played back with subtitles. Visitors are also invited to interact with intelligent musical sculpture able to sonify their voices in real-time.
photo: Florian Voggeneder
photo: tom mesic
photo: tom mesic
Biographies
Credits
Johannes Kepler University Linz (Antonia Ebner, Vasco Fragoso, Emilia Parada-Cabaleiro, Gregor Pechmann, Martin Reiter, Florian Rothkegel, Theresa Rubenzucker, Markus Schedl, Alexander Wallner); Musicians from the Anton Bruckner Private University (Ivan Bykov, Johanna Dömötör, Anna Maria Dörfler, Bernhard Hadriga, Matthias Kronsteiner); Land Oberösterreich
