SuperVision: A Novel Integrated Sensing and Computational Framework to Enable Superhuman Epiretinal Prostheses

Overview

The current gold standard to restore vision to patients suffering from retinal degenerative diseases is through retinal prostheses. However, these FDA-approved implants are limited to low-resolution grayscale images, making object identification and prosthesis adaptation difficult for patients. Furthermore, current studies focus on vision recovery and do not fully leverage state-of-the-art sensing and computational advancements. For the first time, this study develops a paradigm that creates visual capabilities superseding the standard visual acuity. The framework aims to turn patient disadvantages to advantages, effectively creating superhuman vision. It consists of two new integral components in addition to my previous research: 1) a modular plug-and-play (PnP) system to enable the adoption of advanced visual schemes; 2) a tunable optimal transportation theory (OT)-based virtual magnifier to localize and enlarge regions of interest (ROIs) while preserving important features and curvatures; and finally, the resulting images are processed through a real-time image optimization framework and an autoencoder-OT model developed in the previous year’s project. The PnP system provides patients with images at a 120, 180, and 360 degree visual angle. Further, the second generation prototype system was developed to optimize these captured images in both indoor and outdoor scenes. Five computational experiments including visual and area-preserving tests showed that the magnifier enlarged the ROIs with minimal distortion at the varying visual angles. Attention mechanisms and color scheme comparisons demonstrated that the image optimization framework maximized the amount of both spatial and color information displayed to patients.