BeeWell: Developing an AI-Based Bee Health Assessment System Utilizing Computer Vision and Acoustic Signal Processing

Overview

Bees pollinate over 80% of flowering plants, but bee colonies have alarmingly declined by 39.7% annually over the past decade. Traditional methods for monitoring beehives, like human inspection, are time-consuming, disruptive and provide limited information. AI-based methods have been developed to evaluate hive health, but none have combined visual and audio signals. In this study, I developed an innovative Attention-based Multimodal Neural Network (AMNN) to improve bee health assessment. AMNN simultaneously processes paired image and audio data and adaptively focuses on the most important features of each modality. From October 2022 to March 2023, I visited 25 beehives in 3 California apiaries and recorded 100 gigabytes of videos under four health conditions. Videos were processed and annotated to train YOLOv5 and CNN to identify bees in images and audio clips, resulting in 12,440 high-quality records. AMNN integrates visual and auditory aspects of bee behavior for health assessment, achieving an overall accuracy of 92.22%. It significantly outperforms 8 single-signal CNN and RNN models in literature without considerably increasing processing time. Additionally, AMNN improves prediction robustness, achieving over 89.36% F1-score for each health condition. Interestingly, bee sounds were more reliable indicators of health than images in this study. I further integrated AMNN into BeeWell, a system which includes a near real-time streaming mechanism using a Raspberry PI and a web application with bee health evaluation features. BeeWell can help beekeepers monitor their hives remotely, make more accurate decisions to prevent hive collapse and contribute to a healthier ecosystem.

Competition history

• ISEF 2023 Animal Sciences · Entry ANIM042 · Dallas, Texas, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.