TILES: A Bio-Inspired Robotic System for Real-Time Thai Flute Learning and Assessment

Why the Khlui?

The Khlui is a core instrument in Thai music. It expresses Thai identity through delicate sound and microtones.

However, fewer people are learning it. Many schools do not offer Thai flute education, and fewer students continue.

As a result, Thai musical expression is fading.

Problem

Learning the Khlui is complex.

It requires:

Precise finger movements

Controlled airflow

Microtones and pitch control

Expressive techniques such as tonguing and ornamentation

These skills are difficult to measure. Teaching relies on “feeling” rather than data.

There is no learning system for the Khlui. Existing tools only check correct notes and cannot give detailed feedback or evaluate fingering or posture.

In addition:

69% of Thai youth cannot play accurate microtones

80% of schools lack Thai music teachers

No real-time feedback

Because of this, students struggle to improve, and learning becomes slow.

Key Question

How can students learn the Khlui independently while developing musical expression?

Inspiration

This project was inspired by the need to make Thai music more accessible, especially for students without teachers.

Approach

This project introduces a learning system combining AI and robotics.

Buddy Piang-or demonstrates human-like playing using airflow and robotic fingers

A multimodal AI analyzes sound, fingering, and musical context to give real-time feedback

Why It Matters

This system helps:

Reduce learning time from months to weeks

Provide objective feedback

Improve student success

It benefits:

Students and people interested in the Khlui

Schools without music teachers

It supports preservation of Thai musical heritage and wider access.

1. Research & Understanding

We studied how Thai music is taught using textbooks, research articles, and interviews with Thai music teachers. We selected reliable sources to understand real learning problems. We found that students often lack clear, real-time feedback on sound and finger placement.

2. Designing the Solution

We designed a combined system:

TILES (AI learning platform)

Buddy Piang-or (robotic demonstration model)

We chose this design because AI can analyze performance, but a robotic system can physically demonstrate how to correct mistakes. This allows both evaluation and demonstration.

This design is a game changer because it transforms Thai flute learning from a subjective, feeling-based process into an objective, data-driven system.

3. Building the Prototype

We developed three integrated components:

AI Platform: analyzes sound (pitch, rhythm) and finger placement using camera input

AI Tutor (LLM): a language model trained with lesson content that answers questions and explains how to improve in real time

Buddy Piang-or: a robotic system with moving fingers and airflow control to demonstrate correct playing

This allows students to see, hear, and understand how to improve.

4. Testing, Challenges & Iteration

We tested the system through repeated practice sessions.

Background noise affected accuracy

→ Improved sound filtering

Air leakage reduced sound stability

→ Designed an airtight connector for consistent airflow

Robotic movement was unstable

→ Refined motor control for smoother motion

Students needed more guidance

→ Added the AI tutor (LLM) for explanations

5. Data Collection & Fair Testing

Students played the same notes and songs multiple times.

We recorded sound and finger movement data.

To ensure fairness:

Same flute type was used

Same practice tasks were given

Environment was controlled

Summary

By combining AI analysis, an AI tutor (LLM), and robotic demonstration, we created a system that helps students understand what is wrong, why it is wrong, and how to improve.

Results and Analysis

The TILES platform serves as a Gamechanger in Thai flute education, transitioning from subjective assessments to a measurable, data-driven system. By encoding expert knowledge and assessment criteria into the model, the platform provides precise, real-time guidance.

Key Findings:

Increased Pass Rates: The TILES group achieved an 85% pass rate (17/20), significantly outperforming the 55% (11/20) seen in traditional classroom settings.

Drastic Time Savings: Students reached their learning objectives in just 80 minutes using TILES, compared to 250 minutes for the traditional group—saving nearly 3 hours.

AI Evaluation Precision: The system achieved high agreement rates with experts: 93% for fingering, 92% for pitch, and 90% for rhythm.

Professional AI Tutoring: Expert evaluation of the LLM teacher yielded a pedagogical quality score of 3.55/4.

Robotic Performance (Buddy Piang-or): The robot achieved a 98.03% similarity to human play (peaking at 99.82% for the Mi note). It features 99% sealing integrity, a 400ms response time, and a 500ms Air-Cutting system that ensures sharp note articulation by preventing sound overlap.

How the Prototype Works

The ecosystem operates through a Real-Time Feedback Loop (Play → Analyze → Feedback → Improve):

Multimodal AI Engine: Processes real-time audio and visual data to evaluate performance against traditional Thai musical standards.

AI Teacher (LLM): Provides immediate, personalized explanations of mistakes and corrective suggestions.

Buddy Piang-or (Robotic Demonstrator): Uses a tendon-driven mechanism and simulated blowing system to provide a physical, empirical demonstration of correct fingering and articulation.

Discussion

The integration of analysis, explanation, and physical demonstration effectively addresses student confusion. Classroom trials revealed that Demonstration-Based Learning—specifically observing Buddy Piang-or’s movements—is a key factor in boosting student focus and interest. This visual engagement helps students understand complex skills and retain lessons more easily. While high notes (La and Si) show minor deviations due to higher air pressure needs, the robot demonstrates superior long-term pitch stability compared to humans.

Statistical & Analytical Approach

We utilized Digital Signal Processing (DSP) to analyze over 200 note events. Fast Fourier Transform (FFT) was applied for precise pitch detection, allowing us to compare robotic and student performance against human reference models. This rigorous approach confirms that the system provides reliable and accurate assessments for Thai music education.

What We Found

Our results show that learning the Khlui does not have to be slow or confusing.

With AI, students can receive instant feedback not only on whether a note is correct, but also how well it is played, including pitch, rhythm, fingering, and tonguing.

This is something current learning tools cannot do.

Why It Matters

We learned that students improve faster when they receive immediate and clear feedback.

Instead of practicing with uncertainty, they can correct mistakes right away and build skills step by step.

The Buddy Piang-or also helps by showing correct playing physically, allowing students to both see and hear proper technique.

Impact and Future

This project helps solve the problem of limited Thai music teachers by making expert knowledge accessible to everyone.

It also shows that AI can help preserve traditional culture, not replace it.

In the future, this approach can expand to other Thai and global instruments.

Final Message

We are not just teaching notes—we are helping preserve the soul of Thai music.

Improvement

Expand the dataset with more songs, skill levels, and real student performances to increase accuracy. Enhance analysis of posture, fingering, tonguing, and airflow control.

Expansion

Extend TILES to other Thai instruments and adapt it for global wind instruments with similar acoustic principles.

System Upgrade

Improve real-time feedback speed and reduce latency. Integrate the robotic “Buddy Piang-or” for interactive demonstration, correction, and guided practice.

We would like to sincerely thank everyone who supported and guided us throughout this project. First, we thank our families for their encouragement and belief in us. We are grateful to our mentor and advisors for their guidance and valuable feedback. We also thank our teachers and school for providing the resources and environment to develop our work. Special thanks to Thai music teachers and experts who shared their knowledge of the Khlui. We are thankful to all students who helped test our system and gave feedback. Finally, we thank everyone who contributed to making this project possible, as your support has been essential to our journey.

Chapphanrat, Y. (n.d.). Development of Thai music education curriculum and instruction in Thailand [in Thai]. St. John’s University Journal, 281–282. https://sju.ac.th/pap_file/52a110f6b00f437563bbf0365a89b4b7.pdf

Fine Arts Department. (2020). Instructional manual for Lanna folk music [in Thai]. https://www.finearts.go.th/storage/contents/2020/07/file/85B9073UXXwIAhtEcGsYyir68oifx7oq1VQpJdRz.pdf

Jeong, S. H., & Cho, K. J. (2018). A polymer-based soft wearable robot for the hand with a tendon-driven actuation system. Soft Robotics. https://doi.org/10.1089/soro.2018.0006

Pim18. (n.d.). Music. Issuu. https://issuu.com/pim18/docs/music

TruePlookpanya. (n.d.). Poll: Youth and parents support children playing at least one Thai instrument [in Thai]. https://www.trueplookpanya.com/knowledge/content/50175

VERYCATSOUND. (2023). Why is music education expensive? [in Thai]. https://verycatsound.com/blog-womm/

Jeong, S. H., & Cho, K. J. (2018). A polymer-based soft wearable robot for the hand with a tendon-driven actuation system. Soft Robotics, 5(4). https://doi.org/10.1089/soro.2018.0006

ResearchGate. (2017). Frame attachment and the kinematic chain of the index finger [Image]. https://www.researchgate.net/figure/Frame-attachmentfig5_319260102