Tailsense: Classifying Dogs' Emotions Using Barks and Images Based on Breed

Overview

Purpose Dog barks and visual cues are main channels through which dogs communicate their emotional states, such as anger, sadness, and excitement. However, accurately interpreting these is a challenge for humans, needing advanced technological solutions for understanding canine emotions. This study presents a real-time prototype system designed to classify canine emotional states by leveraging a multi-modal approach using audio and image data. Procedure This methodology outlines a systematic approach to developing the TailSense system, leveraging ResNet152 & EfficientNet for dual-modality canine emotion classification. Each step is designed to ensure data quality, model accuracy, and practical deployment, combining in a tool for understanding canine emotional states. The integration of advanced deep learning techniques shows the projects innovation. Built on a Raspberry Pi 5 equipped with an HD camera, a microphone and a 5 inch TFT display, the system is able to take both audio and image input. Results The experimental evaluation showed that the dual-modality TailSense system achieved a validation accuracy of around 90%. The ResNet152 & EfficientNet-based image classifier produced consistent results with a confusion matrix with balanced performance across all emotion categories. Similarly, the spectrogram-based audio classifier had similar accuracy, strengthening the efficiency of the multimodal approach. Conclusion The proposed TailSense system leverages deep ResNet152 & EfficientNet architectures for multimodal emotion classification, achieving an accuracy of approximately 90%. These findings show the potential of deep learning in pet behavioral analysis and lay the groundwork for future research aimed at optimizing this method for further accuracy.

Competition history

• ISEF 2025 Animal Sciences · Entry ANIM006

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.