SmartStrings: A Novel Low-Cost Multimodal System for Intelligent Violin Pedagogy With Real-time Corrective and Sensory Feedback Through Artificial Intelligence

Overview

Playing the violin improves cognitive function, creativity, and brain development. However, access to private lessons is limited by high cost and availability, especially in underserved areas. Online tools exist but often lack personalized feedback and live posture correction, causing poor practice habits and technical deficiencies. To address these challenges, I developed a novel, low-cost Violin Learning Web Application which analyzes performances, deconstructs music into waveforms and key musical components, and evaluates it against a crafted rubric to provide personalized feedback. I also built a 3D landmark tracking system for instant posture assessment, visual and vibrotactile corrective feedback. First, I devised a hybrid deep neural network trained on thousands of self-recorded and cleansed violin audio datafiles, augmented using waveform manipulation techniques and enriched through graphical analysis, zero-padding, and One-Hot Encoding. I applied mathematical modeling using Short-Time Fourier Transform to extract audio features through Mel-Spectrograms, achieving accuracies of 99% and 87% in predicting songs and notes respectively. Next, I deconstructed sound into musical elements like pitch, dynamics, rhythm, texture, and temporal features. I perform comparison analysis using Chromagrams, waveforms, Dynamic Time Warping, Euclidean distance, cosine similarity and cost matrix. The posture corrections, practice suggestions and LLM summarized technical assessments are emailed. Lastly, I improved system capabilities by combining Arduino UNO with Classification Model trained with 99% accuracy to actuate precise haptic corrections. This system bridges the gap between accessibility and lesson quality by personalizing practice plans, feedback, and posture corrections.

Competition history

• ISEF 2025 Technology Enhances the Arts · Entry TECA006

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.