Researchers at the University of Waterloo, Canada, say they have developed a system for generating song lyrics lines conditioned on the style of a specified artist.
According to them, the system uses a variational autoencoder with artist embeddings. The researchers propose the pre-training of artist embeddings with the representations learned by a CNN classifier, which is trained to predict artists based on MEL spectrograms of their song clips.
This work is the first step towards combining audio and text modalities of songs for generating lyrics conditioned on the artist’s style, they stated.
According to them, their preliminary results suggest that there is a benefit in initializing artists’ embeddings with the representations learned by a spectrogram classifier.
The hope is that unusual and creative arrangements of words in the generated lines will inspire the songwriter to create original lyrics. Conditioning the generation on the style of a specific artist is done in order to maintain stylistic consistency of the suggestions.
Such use of generative models is intended to augment the natural creative process when an artist may be inspired to write a song based on something they have read or heard.