Implementing A Hyper-Realistic Physics Engine in The Unity 3D Gaming Software in a Virtual-Reality Based Prosthetic Training & Baseline Tests

Overview

Despite nearly 2 million people in the United States having an upper-arm amputation, 27- 56% of upper-body amputees use a prosthetic arm to regain limited mobility of their arm pre-amputation. 20% of prosthetic users abandon their prosthetic due to improper training and its lack of functionality. Current prosthetic training methods have additional limitations of being costly and restricts the user to a training or rehabilitative facility. Previous research has demonstrated greater accuracy and faster performance in tasks when a user was trained with virtual reality (VR). The aim of this research is to develop a VR environment that can be used as a training platform for prosthetic users, as well as a baseline test for bidirectional sensory feedback. In order to make the virtual environment as realistic as possible, the external MuJoCo physics engine was incorporated into Unity development software to ensure that the physics in the virtual environment mimics the natural world. The MuJoCo physics engine allows virtual objects to have realistic collision and contact events. My research focuses on optimizing MuJoCo so that it interfaces with the Unity gaming software and to attach its properties to game objects in the software. Incorporating MuJoCo physics resulted in game objects tested to have realistic sliding and rolling friction, gravitational forces, velocity and acceleration vectors, density and masses, as well as an ability to be “firm” or “soft” . This re-engineered platform will allow us to conduct grasping and reaching tasks with a virtual, bi-directional neuroprosthesis. GEORGIA Utilizing iPhone Haptic Feedback to Create Refreshable Braille Kevin Jacob Gwinnett School of Mathematics, Science, and Technology, Lawrenceville, GA The expensive nature and overall lack of braille resources has led to an extremely low braille literacy rate among the blind and visually impaired. Literacy in braille is the factor most correlated with employment and educational success of the blind, emphasizing the importance of developing this skill. This project seeks to develop a refreshable braille display using a smartphone’s haptic feedback to represent “raised” braille cells that allows a user to input or scan text as a means to increase the availability of braille resources at an inexpensive cost. The testing performed after the construction of the app was to determine the most optimal settings for a visually impaired user. A combination of a 3D printed guide and heavy levels of haptic feedback were hypothesized to result in the most accurate readings by the user. These variables were tested separately after the app was coded in Swift using the XCode IDE. Random strings of 60 characters were generated and a box was used to obstruct vision, allowing for testing of the app. After the trials were concluded, the original hypothesis was deemed correct and resulted in high accuracy rates of 85% and 90%. This project will increase the equity of education and allow the visually impaired to learn the same literacy skills that are expected of those that are not