Where'd Those Swimmers Go?

Overview

Objectives/Goals A camcorder filmed a pool with swimmers above and below the water, and a computer system determined when swimmers were in danger of drowning. Methods/Materials The initial approach used a FLIR T640 infrared camera in attempting to distinguish hot swimmers from their cooler surroundings--water and objects. However, experimentation showed that reflected sunlight could often falsely make the surroundings appear hotter than swimmers. Consequently, an alternative approach was implemented using a Sony PMW-EX1 Camcorder. Streamed video was captured at two frames per second and processed in real-time using a Python program. To ignore objects, only closely-spaced pixels with a skin hue were clustered together. Thresholds for a cluster's perspective-adjusted area as well as the absolute number of pixels determined whether a cluster was significant, i.e., corresponded to an above-water swimmer. Once clusters were initialized in a frame, the locations and constituent pixels for the significant clusters were simply updated in subsequent frames, enabling swimmers to be tracked. The updating procedure was faster than clustering. When a swimmer submerged, the corresponding cluster was eliminated because its area and/or number of pixels dropped below the thresholds. In response, the program scanned new frames and re-initialized the clusters to see if the swimmer had resurfaced. For real-time implementation, the program code was optimized for speed and also rewritten to run on multiple computers and cores. A master process running on one core launched slave subprocesses. The slaves analyzed multiple images simultaneously, performing the skin pixel classification and clustering. The master used this data to update clusters and determine when swimmers were underwater. Results Using hue to classify pixels worked well, enabling the system to ignore objects in the pool. In identifying significant clusters, it was necessary to adjust the cluster area because a swimmer's apparent area depended on his distance from the camera. With multiple cores and speed-optimized code, the system was able to process two frames per second with a maximum five second latency. Conclusions/Discussion The system accurately tracked swimmers in real-time and determined when they submerged or resurfaced. It monitored the duration that a swimmer was underwater and generated an alert if too long a time elapsed without the swimmer resurfacing.