Using Artificial Intelligence and Computer Vision in Creating an Operating System for Head-Mounted Displays

Overview

Computers have always been a huge part of our lives, whether in desktop or mobile applications. Some think of the mobile as the future, yet with the computational power constantly evolving it’s clear that the world is going to different platforms especially with the smartphone market being close to saturation by most predictions. Wearable computers, especially head-mounted displays are introduced to create augmented and virtual reality (AR/VR) .Yet with the possibility to create a device that’s powerful enough to satisfy both needs, resources are wasted and It’s clear that this waste will grow greater by the arrival of the first AR and VR products into market, and current technology used in these devices does not allow a great window of multitasking because of the lack of OS support. This project aims to create a Linux distribution that is capable of running Virtual and Augmented reality applications/Environments, while keeping legacy support to current critical desktop apps, using a stereo camera and the OpenCV library for depth detection and applying a feed forward artificial neural network for the hand region detection and factorial Hidden Markov models for gesture recognition. The Depth image is also going to be used for creating a 3D map on the cloud using OpenGL along with the GPS coordinates. A stereoscopic desktop environment is made based on GNOME, and transformations using OpenGL are applied to normal Desktop applications to add the depth effect and make them appear floating in front of the user. A Head-mounted display capable of tracking the user’s head movement to achieve desired emulation for both VR and AR settings is made so it would also provide the first-person depth map, thus achieving ubiquitous computing and convergence via wearable computers.

Competition history

• ISEF 2014 Embedded Systems · Entry CS012

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.