Neural networks can learn and apply styles from music genres with ChordGAN

Conan Lu

Overview

ChordGAN is a generative adversarial network that transfers the style elements of music genres. As a field, symbolic music style transfer has presented exciting ways of interpreting music with different genres. Early attempts have faced the challenge of preserving content features while changing style features to fit a target genre. ChordGAN seeks to learn the rendering of harmonic structures into notes by embedding chroma feature extraction within the training process. In notated music, the chroma representation approximates chord notation as it only takes into account the pitch class of musical notes, representing multiple notes collectively as a density of pitches over a short time period. Chroma is used to improve the consistency of transfer, in addition to conditional GAN architecture that parallels image-to-image translation algorithms. Pop, jazz, and classical datasets were used for training and transfer purposes. To evaluate the success of the transfer, two metrics were used: Tonnetz distance, to measure harmonic similarity, and a separate genre classifier, to measure the transfer style fidelity. Given its success under these metrics, ChordGAN can be utilized as a tool for musicians to study compositional techniques for different styles and generate music from lead sheets.

Video

From the student

I started playing the piano in first grade, and the oboe in sixth. When I practiced as a child, I amused myself by reimagining the compositions I was learning in other styles. The four-note Alberti bass pattern of classical music would be worked into a Top 40 radio hit, jazz chord configurations would be implemented for Mozart's simplest pieces.

What would a Bach piece written by a pop composer sound like? I wasn't aware at the time, but these questions eventually became the official starting point for my research.

Many years later, in 2019, I attended COSMOS at UC San Diego in Cluster 9: Music and Technology. One of the mentors had completed substantial research in using AI systems for music.

At COSMOS, I had the opportunity to delve into the technology behind music production. I remained interested in creative music AI afterwards, so I reached out to my mentor independently in hopes of working with him on more substantial projects. With no prior research experience, I spent the next few months becoming more familiar with music technologies and machine learning basics.

For over two years, my mentor and I have tackled the problem of style transfer: how different digital representations of music can improve the way computers learn to compose. The final model was named "ChordGAN."

Today, I (along with the scientific community) have the privilege of knowing what a Bach piece in the style of Taylor Swift's country era sounds like.