Real-Time Disaster Search and Rescue System Utilizing Semantic Segmentation and Object Recognition onboard Fixed-Wing UA

Overview

Traditional search-and-rescue (SAR) operations rely heavily on manual aerial interpretation and often fail under low-visibility or communication-denied conditions. This project aims to address the engineering challenge of performing multimodal survivor detection under extreme pixel sparsity and strict edge power constraints, without reliance on cloud infrastructure. A fully onboard artificial intelligence system was engineered on an NVIDIA Jetson Orin NX platform with a custom TensorRT-accelerated inference pipeline. The vision subsystem integrates a modified YOLOv11 detector optimized for sub-10-pixel aerial human targets and a custom Swin Transformer–based terrain segmentation model. Both networks were restructured using FP16 quantization, layer fusion, and pipeline parallelization to maximize performance per watt and minimize latency. Visual inference was integrated with a four-microphone array for voice direction-of-arrival estimation and a wireless network interface for RF presence detection through a custom weighted multimodal fusion architecture. Engineering validation emphasized throughput, latency, and power efficiency under full multimodal load. The system achieved a 95th percentile throughput of 156 frames per second while maintaining a 95th percentile power draw of 25.0 W, satisfying real-time flight constraints. Integrated mission testing confirmed synchronized cross-modal detection and successful human identification at 200 feet under both visible and partially occluded conditions. This test achieved a detection rate 2.6 times greater than that of a single-modality (vision-only) test. By demonstrating that transformer-based segmentation, real-time object detection, and acoustic/RF sensing can operate concurrently within strict edge power budgets, this work establishes a scalable systems-engineering framework for resilient disaster-response UAV platforms.