A deep neural network usually contains a very large number of parameters – millions or even billions – that are learned during the training; a complexity that’s needed for the nonlinear internal representations of the input data. This installation visualizes some aspects of the inner life of a deep neural network for music composition called Ricercar that is being developed at Ars Electronica Futurelab.
Ricercar is trained with 25000 pieces of music and can compose music. In this case, however, the network will not compose, but be fed with an existing piece of music as input.
We observe the reaction of the neurons in each layer as the music continues, showing us every moment in the piece in which neurons are activated and react to it. Finally, a so-called “similarity matrix” shows the similarity between the activated neurons at any point in time with any other. In other words, it shows reveals which parts in the piece are similar as they produce the activation of similar neurons.
The result is a novel way of visualizing music, which shows the higher-level structures of a piece of music (e.g. its repetitive structures). It also shows us, once again, that music is not only pleasant to hear, but also contains beautiful hidden visual structures that can span the entire piece.
