LeAF: Leveraging Deep Learning for Plant Anomaly Detection for Farmers

Overview

Farmers face numerous challenges in crop cultivation, particularly in monitoring and maintaining plant health. Plant anomalies such as pests, diseases, and weeds negatively impact plant health, leading to decreased crop yield. Over 40% of global crop production is lost to plant anomalies, costing $220 billion annually. As crop production increases in response to rising global food demand, manual surveillance for plant anomalies becomes increasingly difficult, resulting in excessive and indiscriminate use of fertilizers and pesticides. This not only escalates costs but also heightens consumer concerns about chemical residues, runoffs, and emissions. LeAF is a comprehensive system to survey crops in real-time by leveraging recent advancements in multimodal Artificial Intelligence (AI) with Convolutional Neural Networks (CNNs) and Large Language Models (LLMs). LeAF achieves six objectives: (1) utilizing CNNs to analyze robot camera feeds for plant anomalies with bounding box detection and classification, (2) employing plant stem identification to attribute anomaly data to specific plants and create field maps, (3) integrating a domain-specific LLM to provide farmers with optimal treatment suggestions, (4) supporting question-answering on farming techniques, (5) offering estimates on strategy-effectiveness, cost-savings, and environmental impact reduction, and (6) deploying a custom-made BRANCH robot (Budget-friendly Robot for Agricultural Nonintrusive Crop Photography) at local farms that costs under $500. This end-to-end solution addresses the challenges faced by farmers, empowering them with actionable insights to enhance crop management efficiency while minimizing environmental impact.