Instrumental Sound Separation Using Compressed Machine Learning Models

Overview

To assist amateur music performers, I have developed a general-purpose instrumental sound separation method that introduces "selective inference" and makes it possible for a large number of instruments to be separated with high accuracy. It should be natural that the separation model becomes simple if we only consider a specific group of instruments. We divide the separation task into several such simple models, and we can separate the sound of many instruments with high accuracy by combining those small models. To achieve this, we focused on the FCN (Fully Convolutional Neural Network) structure used in semantic separation techniques that have already demonstrated high performance in machine learning and applied indirect inference using spectrograms to FCN. In addition, several innovations were made to maintain high separation accuracy even when the model is compressed. First, the output method was modified to account for the unobstructed nature of the sound signals. Second, we make the rectangle window on the spectrogram plane longer in the time axis direction as a sound signal does not usually change so rapidly. Comparative experiments show that the new output method to incorporate the non-occlusive nature of sound signals improves the separation accuracy. A longer window shape in the time axis direction does not contribute so much. These results indicate that "selective inference" is useful to increase the number of separable instruments, improve consistency with the original sound, and reduce inference time while maintaining the same high separation accuracy as existing methods for models with good accuracy.

Awards (2)

• Association for the Advancement of Artificial Intelligence: Honorable Mention
• Association for the Advancement of Artificial Intelligence: AAAI Membership for the School Libraries of All 8 Winners (in-kind award / part of 1st-3rd prize and honorable mentions' prize)

Competition history

• ISEF 2022 Embedded Systems · Entry EBED016 · Atlanta, Georgia, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.