Argus III: A Novel Image Optimization and Augmentation Framework To Enable an Improved Patient Experience for the Next Generation Epiretinal Prosthesis

Overview

Retinal prostheses provide a promising solution to restore vision to the 200 million people who suffer from retinal degenerative diseases worldwide. However, current FDA-approved implants are limited to low-resolution grayscale images, making object identification and localization difficult for patients. Furthermore, patients will face a steep learning curve to adapt to the prostheses. To address these challenges, this study develops a novel methodology framework that consists of three integral components: 1) an optimal transportation theory (OT)-based virtual magnifier to localize and enlarge regions of interest (ROIs) while preserving important features and curvatures; 2) a real-time image optimization framework to encode the maximum amount of spatial and color information to patients through attention mechanisms as well as color scheme comparisons; and 3) an autoencoder-OT model to augment the optimized images. Computational experiments through distortion maps showed that the magnifier enlarged the ROIs with minimal area and angle distortion. Further, users were able to select important features and optimize ROI densities through a “digital knob” user interface. In contrast to current schemes, the image optimization framework demonstrated better visual quality, was computationally efficient (<380 ms on tested cases), and allowed for optimal color mapping through comparison studies. A prototype processing system confirmed the effectiveness of the proposed optimization framework over current prostheses. Finally, the AE-OT model augmented images from 6 datasets to generate an image library for patient training. This research offers an accurate, scalable, and optimized architecture that will enable the next generation epiretinal prosthesis.

Awards (4)

• Second Award of $2,000 $2,000
• Oracle Academy: Award of $5,000 for outstanding project in the systems software category. $5,000
• Shanghai Science Association for Young Talents: Science Seed Award
• Patent and Trademark Office Society: Second Award of $500 $500

Competition history

• ISEF 2022 Systems Software · Entry SOFT014 · Atlanta, Georgia, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.