The new LyricJam system will help artists compose new lyrics that go well with the music.
The goal of our research is to develop a system that can generate lyrics that reflect moods and emotions, expressed through various aspects of music. We are talking about chords, used instruments, tempo. We aimed to create an instrument that musicians could use to draw inspiration for their own songs.
Olga Vechtomova, study author
Essentially, LyricJam is an artificial intelligence system. From the user’s point of view, the application is very simple: the music artist plays live music, and the system displays lines of text that he generates in real time in response to the music he hears. The generated lines are saved.
The system, created by the researchers, works by converting raw audio files into spectrograms. Deep learning models are then used to create lyrics that match the music they were processing in real time. The architecture of the model consists of two variational autoencoders, one of which is for studying the presentation of musical sound, and the other for studying texts.