Modality Matched Sensory Smart Skin System to Restore Tactile Sensation for Amputees With Adaptive Visuo-Tactile Predictive Texture Recognition

Overview

Over 57 million people worldwide live with an amputation, and more than 80% of them live in developing countries, where access to therapeutic remedy is especially constrained. To date, work done on improving mobility of prosthetic hands has been mildly successful, but a daunting challenge remains in restoring the sensory function to forearm amputees. Our ability to manipulate objects relies fundamentally on sensory functions, such as the sense of touch, feeling the temperature of objects, feeling the force being applied and feeling the texture of materials, which are still lacking in prosthetic advances. Texture recognition plays a pivotal role as the characteristics of a material's surface significantly affect grasping. Analysis of surface properties, such as vibration and friction during contact, enhances texture recognizing capabilities. While ML techniques provide effective tools, the limited availability of labeled texture samples further complicates the process of identifying previously unseen textures. To solve this problem for upper arm prosthetics, this research develops a novel multimodally-sensing cost-effective prosthetic hand system integrated with digitally embedded smart skin that restores not only active mobility to arm amputees, but also re-equips them with critical sensory feedback via the Mechanotactile feedback along with a predictive visuo-tactile texture recognition ML model using visual and haptic features to over come the limited availability of texture database samples. The sensory feedback system designed in this work demonstrated precise restoration of tactile sensation for amputees, and can enhance robotic texture perception, enabling safer interactions and improved grasping strategies.

Competition history

• ISEF 2025 Embedded Systems · Entry EBED007

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.