Creating an Artificial Intelligence Neural Network to Critique Expressive Piano Performance

Overview

Classical music education is a complex field requiring the teacher to evaluate a student based on audio and visual information. However, this approach can be error prone and biased. This project proposes a framework of a 3-component piano performance analyzer that can provide feedback to students without the input of an instructor. The core component for analyzing the performance was focused on in this project. Training data was synthesized from existing piano performance datasets by randomly generating errors within the pieces. Two classes of errors were categorized, with simple errors describing one note, and high-level errors describing a series of notes. To predict these errors, multi-class and multi-label networks composed of difference modules and inception modules inspired by GoogleNet were created using 1D Convolutions. The model for predicting simple errors was able to reach an accuracy level of 97%, where accuracy was measured in how closely the output of the model matched the error codes for the piece. The accuracy of the high-level error model peaked at 90%. The results validate the current architecture of the analyzer component. Including the other components for generating an optimal performance and translating the student performance would complete the full model. For the analyzer component, future models could experiment with using a deeper network or combining it with a large language model to generate text descriptions of critique.

Awards (1)

• Third Award of $1,000 $1,000

Competition history

• ISEF 2024 Technology Enhances the Arts · Entry TECA016 · Los Angeles, California, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.