BeeWell: Developing an AI-based Bee Health Assessment System Utilizing Computer Vision and Acoustic Signal Processing

Overview

Bees pollinate over 80% of plants, but bee colonies have been experiencing a devastating 39.7% annual loss over the past 11 years. To address this pressing issue, BeeWell, an innovative deep learning-based system was developed from data acquisition through Raspberry PI and a web application with near-real-time streaming and bee health evaluation features. The innovative system is the first to integrate bee health assessment with bee object detection using paired visual and audio data in research. I collected the paired data from 25 beehives in 3 apiaries in California, resulting in 4 datasets with 12,440 high-quality records. Bees were detected from images and audio clips with 98.60% mean average precision by YOLO and with an accuracy of 94.72% by CNN model coupled with Chroma feature extraction. The novel multimodal deep neural network that effectively combined visual and audio signals achieved a good accuracy of 92.61% for bee health assessment, outperforming VGG16 visual model at 69.89% accuracy and VGG16 with STFT audio model at 81.25% accuracy. Notably, image signals captured 13.21% of cases missed by audio, while audio signals captured 21.21% of cases missed by image. The study shows that visual and audio signals can complement each other, audio is a more reliable indicator of hive health, and the sequential patterns in the audio signal might not be informative for predicting bee health. With BeeWell, beekeepers can take prompt action to protect their colonies, ultimately leading to a healthier and more sustainable ecosystem.

Awards (1)

Category Award: 2

Competition history

CSEF 2023 Zoology · Entry S2203

Resources

Related projects

ISEF · 2023