AI Sentry

Project Origination & Purpose

The summer before I started high school, nearly five years ago, my dad introduced me to the amazing sport of paintball. Ever since, I have had the idea to create an autonomous-paintball machine, capable of playing this game. After having gained  experience from my last two science fair projects, I now felt comfortable pursuing this project. After approximately 1500 hours, over my summer vacation and many months of school, it was brought to life as AI Sentry.

AI Sentry is a waterproof, customizable, and user-friendly sentry that is suitable for many PAN and TILT applications, regardless of the environment or weather conditions. Currently, PAN and TILT systems are used commercially for devices that perform single tasks (e.g. PTZ security-camera that can only track and record). Unlike these products, AI Sentry is fully customizable, allowing users to screw-in any desired attachment (spotlight, camera, paintball marker, …) into the fixture plate, making it suitable for countless applications. Attach a spotlight and camera, and it can serve as an advanced security-system. Attach a paintball marker, and it can serve as an autonomous-paintball machine for recreational purposes. The AI Sentry could do all this, and more!

Design Overview

I) Hardware

The sentry was designed using Autodesk Inventor (3D-CAD Software), and was built using a 3-axis CNC-Machine (to custom make the metal parts), TIG-welder, and 3D-printers.

PAN and TILT motors allow the sentry to engage with any target within the cameras’ field-of-vision. Four metal base-beams allow the motors to rotate without the sentry rocking or falling over.

A waterproof electrical-box was constructed, to serve as the center-processing unit of AI Sentry. It contains the Jetson computer which runs the software, the power-supplies that power the electronics, and the Control-PCB used to control the attachments and the motors.

II) Software

The sentry’s software is split up into the camera program (for detecting targets) and the motor program (for locking and engaging with the targets). Both were programmed in C++, on the Linux-Ubuntu operating system.

The camera program was created first. Using the stereoscopic-cameras and skeletal body-tracking software, it finds the 3D-position and velocity of each target in the frame. It then instantly calculates the number of degrees (PAN and TILT degrees) needed for both motors to rotate, in order for the attachment to correctly center itself on the target.

The motor program was created second. Using each target’s PAN and TILT degrees, the program goes through a 4-state loop to easily lock-on and engage with the target. Each target is engaged for 500ms (which can be customized). Depending on which attachment is mounted, the sentry will engage with the target accordingly. For example: i) if a spotlight is mounted, the sentry will shine light directly on the target, ii) if a paintball-marker is mounted, it will aim ahead and above the target, to account for the paintball and target’s velocity. If all the targets have already been engaged, it cycles back to the first target.

Results

Currently a flashlight-attachment is used to simulate how AI Sentry can detect, lock-on, and engage with a target. For each target, the 4-state loop is completed in approximately 250 ms, or even faster, depending on the position of the last target. As a result, AI Sentry is now ready for field testing, which will be done at my local paintball field (not open during the winter), after my exams.

Conclusion

AI Sentry is a customizable (ability to mount any attachment), user-friendly (simply screw-in the desired attachment, and plug in the connector!), and waterproof sentry suitable for various applications. The user can utilize AI Sentry for whichever application suits them (can interchange between an advanced security-system, an autonomous paintball machine, and much more in the future!).

YOLO will be used to train AI-models to detect objects based on image datasets provided by the user, so AI Sentry can also detect illegal-activity within a crowd by scouting for weapons and other dangerous items.

“Zoom”-lensed stereoscopic cameras will provide higher video-resolution at farther ranges, to increase AISentry’s detection range. These will work side-by-side with LiDAR/Radar sensors for smaller-object detection at longer ranges.

AISentry will recognize the mounted-attachment to prevent any unauthorized/illegal attachments from being used.

AISentry will be sold commercially, not only for security-systems and recreational purposes, but eventually for law-enforcement and drone-defense applications.

Acknowledgments

Over the past three years, my father has supported me in developing the skills I have used in my science fair projects, which have become more complex as I’ve learned more each year. All of these experiences have allowed me to build this year’s next-level project. In addition, I am especially grateful that he taught me and trusted me to use a 3-axis CNC-machine.

“Adaptivity as fast and as simple as possible | Autodesk Inventor” youtube.com. By Tech3D. Last Accessed: 3/10/2015

https://www.youtube.com/watch?v=rAHKdsjGdNA

“Metric Tap Chart” armstrongmetalcrafts.com.

https://armstrongmetalcrafts.com/Reference/MetricTapChart.aspx

“Tap And Drill Size Chart” americanfastener.com.

https://www.americanfastener.com/tap-and-drill-size-chart/

“Hannibal Pre-Ream Drill Size Chart” hannibalcarbide.com.

https://www.hannibalcarbide.com/wp-content/uploads/2024/02/pre-ream-drill-size-chart.pdf

“Dowel Pin Installation Design Tolerance Table Chart” engineersedge.com.

https://www.engineersedge.com/dowel_pin.htm

"What Is An Optical Breadboard?” onepointesolutions.com. Last Accessed: 2/4/2021.

https://www.onepointesolutions.com/blog/what-is-an-optical-breadboard/

Overview Artemis Direct-Drive Motor Datasheet

Artemis - Overview Ltd (Datasheet Protected under NDA)

“Tight-Tolerance Multipurpose 6061 Aluminum Rod - ¾” Diameter” mcmaster.com.

Tight-Tolerance Multipurpose 6061 Aluminum Rod, 3/4" Diameter | McMaster-Carr

“Permanently Lubricated 440C Stainless Steel Ball Bearing -¾” Shaft Diameter” mcmaster.com.

Permanently Lubricated 440C Stainless Steel Ball Bearing, Sealed, Trade Number R12, for 3/4" Shaft Diameter | McMaster-Carr

“Rechargeable Flashlight” amazon.com.

Rechargeable Flashlight 300,000 Lumens, 2 Pack Super Bright LED Flashlight

“CLINKO 24-pin Waterproof Connector” made-in-china.com. (bought: aliexpress.com)

https://vi.made-in-china.com/co_cnlinko/product_Cnlinko-M24-IP67-Signal-DC-5A-Electronic-Female-19pin-Connector-Waterproof-Push-Pull-Quick-Connector-250V-Soldering-Connection_osyoeeeug.html

“CLINKO LP16 Waterproof Type-C Connector” aliexpress.com.

CNLINKO LP16 Waterproof Type-C Connector Charging Power 10Gbps High-Speed Data Transfer for Outdoor/Industrial/Marine Use - AliExpress 13

“CLINKO M24 USB3.0 Female Waterproof Connector” aliexpress.com.

CNLINKO M24 USB3.0 Female Waterproof Connector USB Metal Socket Connector 1.8A Receptacles Jacks for Video Audio Data Transfer - AliExpress 13

“Waterproof SMA Female to Female” aliexpress.com.

10pcs/Lot Waterproof SMA Female Jack to SMA Female Bulkhead Straight Adapter for Radio Antenna 50 Ohm RF Connector BEVOTOP - AliExpress 13

“RJ45 Male to Female Cables with Waterproof Cover Shield” aliexpress.com.

RJ45 Male to Female Cables with Waterproof Cover Shield CAT6A Date Transmission Line for Front End Chassis Metal Panel Socket - AliExpress 13

“HD 4K DisplayPort DP Cable Connector” aliexpress.com.

HD 4K DisplayPort DP Cable Connector Solderless for TV Video Interface, AVSSZ Waterproof DP Docking Extension Line Female Module - AliExpress 44

“AC Power Socket 3 Pins” aliexpress.com.

5/20Pcs AC Power Socket 3 Pins Male Plug Copper Pin AC-04 Electrical Socket Mount Inlet Connector - AliExpress 13

“Jetson AGX Orin J30 GPIO Expansion Header Pinout” jetsonhacks.com.

NVIDIA Jetson AGX Orin GPIO Header Pinout - JetsonHacks

“Body Tracking Samples” github.com.

https://github.com/stereolabs/zed-sdk/tree/master/body%20tracking

“Electrical-Box Frame” amazon.com

https://www.amazon.ca/VEVOR-Electrical-Weatherproof-Wall-Mounted-Electronic/...

“Coordinate Frames” stereolabs.com.

https://www.stereolabs.com/docs/positional-tracking/coordinate-frames

“How To Install ZED SDK on Linux” stereolabs.com.

https://www.stereolabs.com/docs/development/zed-sdk/linux

“ZED Link Quad on AGX Orin” stereolabs.com.

https://www.stereolabs.com/docs/embedded/zed-link/quad-jetson-orin-agx-devkit-setup

“Install the ZED Link driver” stereolabs.com.

https://www.stereolabs.com/docs/embedded/zed-link/install-the-drivers

“Setting up the ZED X One Stereo” stereolabs.com.

https://www.stereolabs.com/docs/cameras/zed-x-one/zed-x-one-stereo

“Stereo Calibration on Zed X One tips” community.stereolabs.com.

https://community.stereolabs.com/t/stereo-calibration-on-zed-x-one-tips/10018