How to compose with MuseNet a piece of music in the style of Mozart

We tried for you how to compose music with the Deep Neural Network "MuseNet" from OpenAI and made a step-by-step guide based on it. If you want to learn more about composing music using AI, you should visit the Open Soundstudio at the Ars Electronica Center and participate in the "MuseNet Special" workshop.

| | |

MuseNet is a Deep Neural Network that can create four-minute musical compositions with ten different instruments and can combine styles from Country to Mozart to the Beatles. “MuseNet was not programmed with our understanding of music, but has discovered patterns in harmony, rhythm and style itself by learning to predict the next character in hundreds of thousands of MIDI files,” the OpenAI developers write about their project. “MuseNet uses the same unsupervised multi-purpose technology as GPT-2, a large-scale transformer model trained to predict the next character in a sequence, whether audio or text.

The developers want to make MuseNet available to musicians and non-musicians alike. Following this approach we have tried MuseNet and have written down the individual steps for you. With this manual you can try out MuseNet yourself and compose your first own piece of music.

If you want to learn more about MuseNet and Artificial Intelligence, visit our theme weeks “Artificial Intelligence – The Revolution Behind the Hype” until February 2nd at the Ars Electronica Center. Not only will you be able to learn more about composing music with the help of AI at the MuseNet Special (Open Sound Studio), but you can also delve deeper into the topic of artificial intelligence in expert talks, workshops and deep-space presentations. Those who missed the theme weeks should take a look at the new exhibitions and especially the Understanding AI Exhibition, which is fully dedicated to AI and its impact on us as a society.

1. Open MuseNet

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

First we open MuseNet, via the link below: https://openai.com/blog/musenet/. What seems to be a blog post is at the same time a kind of manual for MuseNet and a bit hidden the program itself.

2. The Simple Mode

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

Under the item “Try MuseNet” you can start right away in the “Simple Mode” that is displayed by default: Here you can listen to numerous random, untreated samples pre-generated by the team. The user can choose composer or style, an optional beginning of a famous piece, and thus start the generating process. We have chosen to listen to music in the style of Mozart, starting with Adele’s Someone Like You.

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

MuseNet then produces for example the following three variants (in a shortened version):

3. Composing in Advanced Mode

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

In the advanced mode (“Advanced Settings”) you can interact directly with the model and generate your new, completely unique pieces. It may take a little longer to complete, but in less than a minute the program can compose a four-minute piece of music.

MuseNet generates four alternatives. If you don’t like these, you can click on “reset” or “update” to create new variants.

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

In the advanced settings it is possible to select not only the instruments that MuseNet should use, but also to change the number of “tokens”, the characters.

We first changed them from the default 225 Tokens to 400. This sounds like this, for example:

Now we play with the sound of our piece of music by changing the instruments and adding to piano, strings, drums and bass.

This shows one limitation of MuseNet: The instruments you wish to have are only suggestions, not requirements. MuseNet generates each note by calculating the probabilities of all possible notes and instruments. Although the model shifts to make my instrument choice more probable, there is always a chance that it chooses something else.

In our case, MuseNet does not find the pairing of Mozart, Adele and our chosen instruments so appropriate and decides to compose with piano only.

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

But we want to try a variant with several instruments and therefore change the style from Mozart to, for example, the Beatles, where instruments such as guitar or bass actually appear. So we choose bass, piano, drums and guitar and see what happens:

4. Download the Composition

If you are satisfied with a generated variant and don’t want to change anything else, you can download the piece. If I just want to save the piece of music for my personal use, I choose the Mp3 format.

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

For musicians, or those who want to become musicians, I can also save the file as a MIDI file (Musical Instrument Digital Interface, in short: MIDI is an industry standard for the exchange of musical control information between electronic instruments, such as keyboards or synthesizers), for example to continue composing in a program like MuseScore by myself. With the help of this open source program I can create or modify sheet music and create compositions just like a professional.

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

5. Composing with Custom Melodies

MuseNet by OpenAI at Ars Electronica Center’s Open Soundstudio, Credit: Ars Electronica / Magdalena Sick-Leitner

Advanced users can also upload their own melodies or copyright-free melodies as MIDI files and continue working with them, i.e. use them as intros.

Notes

A few comments about MuseNet: We have already mentioned a few restrictions. We also have the feeling that classical music might work better than pop because MuseNet is filled with a lot of classical music data. Piano also tends to work better or is used more often than other instruments.

Also, according to OpenAI itself, MuseNet has a harder time with unusual pairings of styles and instruments (such as Chopin with bass and drums). The generated works become more natural if you choose instruments that come closest to the composer’s or band’s usual style.

Have fun trying it out!

If you enjoyed the compositions of MuseNet, we would like to recommend the following video: MuseNet played an experimental concert on April 25, 2019, which was live streamed on the Twitch channel of OpenAI without anyone having heard the compositions before. You can listen to the whole concert on Twitch or Youtube:

Train self-driving cars, produce music using AI algorithms, experience a computer piano that can record and precisely reproduce pieces played by human pianists, or learn more about a neural network that composes music à la Beatles. From January 14 to February 2, 2020, the Ars Electronica Center will be devoting itself to the topic of “Artificial Intelligence – the Revolution behind the Hype” with expert discussions, special guided tours, workshops and deep-space presentations. The entire program: https://ars.electronica.art/center/en/theme-artificial-intelligence/

To learn more about Ars Electronica, follow us on FacebookTwitterInstagram et al., subscribe to our newsletter, and check us out online at https://ars.electronica.art/news/en/.