A Computer Vision Based Quality Monitoring System for Safety Critical Assembling with Artificial Intelligence

Overview

Objectives: The 2024 Alaska Airlines Flight 1282 midair door separation highlights how missing components can lead to a catastrophic failure in an aerospace system. The National Transportation Safety Board (NTSB) investigation indicates that bolts retaining the door plug were missing before the flight. To prevent future accidents, this project proposes a computer vision based quality monitoring system using artificial intelligence (AI) to detect missing components during airplane assembly to capture the problem up front and hence enhance aviation safety. Methods: The system utilizes a camera to capture real time assembly images at 10 Hz. Then, these images are transmitted through Wi-Fi to a cloud-based AI model (CNN or ResNet trained on 15,176 images) that detects missing components. Upon identifying a defect, the system immediately triggers an alert with a human machine interface, allowing the worker to instantly resolve the issue. Frame by frame model performance testing was performed with a confusion matrix in order to calculate the accuracy and false positive. Integration testing (simulates real time assembly) was performed on a 1:1 scaled model plane door where the operator intentionally didn’t install one or multiple bolts to simulate an assembly defect. To succeed, the system would have to correctly identify the class: complete, missing, irrelevant. Using combinations of 4 variables (lighting, perspective angle, distance, and bolt color) and repeating it 10 times to ensure robustness. Results: Four models were trained using a combination of part color (red or silver) and model backbone (CNN or ResNet). The frame by frame testing indicates that the ResNet models (92.63%) outperforms the CNN models (89.85%) in average accuracy for both colors. Therefore, the two ResNet models were chosen for integration testing. Under variable lighting conditions and perspective angles, ResNet Red achieves a 96.25% accuracy by reporting correct and missing installations, which outperforms ResNet Silver (73.75%) by 22.5%. Conclusion: Overall evaluation demonstrates that the AI system can be used to reliably detect missing components during airplane assembly and hence improve aviation safety. Integration testing shows that the silver bolts have a similar color as the door background and hence lowers detection accuracy. Further experiments suggest that replacing silver with red bolts allows for better visual distinguishability and improved detection accuracy. This proposed solution can be integrated into many other applications such as the automotive industry.

Competition history

• CSEF 2026 Applied Mechanics (Junior Division) · Entry J-02-21

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Engineering projects
• CSEF Engineering projects
• Every CSEF 2026 project