A Universal Music Model: deep learning architectures for automatic arrangement, production and generation of songs starting from partial information encoded in mel-spectrograms.
| Componente | Categoria |
|---|---|
| Stefano Leonardi | Tutor di riferimento |
Over the past seventy years, much research on automatic music generation has been conducted. From melody creation to chord progression, computer scientists, along with music experts, have applied increasingly sophisticated computational methods to the various tasks that make up the creative process.
Quite recenty, the impressive advances in machine learning - as well as the sharp decrease in computational costs - have led to a leap forward in automatic music production. After years of limited results, at the beginning of 2020 OpenAI researchers released a novel deep learning architecture that is able to effectively generate new songs. Trained on more than one million songs, OpenAI Jukebox has successfully addressed many tasks that the MIR community has been chasing for decades.
Following this new promising path, in this project we want to design and train a transformer-based maching learning algorithm that would be able to produce a convincing sound landscape around an acappella song given as input. To the best of our knowledge, we are the first to address this specific generation task in the music field. Using the mel-spectrogram representation of a large corpus of songs, we will teach our model to complete spectrograms where only partial information - i.e. the vocals - is available.