Safer Hands in Hazardous Environments: Object-Centric Reinforcement Learning for Sim-to-Real Dexterous Manipulation
ISEF · 2026 Software Design
Overview
Robotic manipulation of hazardous or sensitive objects is essential in laboratory, industrial, and remote environments, but remains challenging due to object variability. While teleoperation enables human control in these settings, it requires continuous attention and limits scalability. This project presents an object-centric reinforcement learning framework that enables a robot to learn manipulation tasks from a short human demonstration and execute them autonomously. Using NVIDIA Isaac Sim, the system extracts object-level task objectives and physical outcomes rather than explicitly imitating joint or trajectory motions, allowing the policy to reason about desired object behavior. Reinforcement learning then refines the policy under randomized physical conditions, thereby training the policy's robustness to variations in object shape, mass, friction, and contact dynamics. The learned policy is evaluated on multiple object-handling tasks involving previously unseen variations of objects. Performance is measured using task success rate, completion time, safety violations, and required human intervention. Results show that the object-centric approach improves adaptability to novel objects while reducing human workload compared to teleoperation-only control. By centering learning on object-level task outcomes and leveraging large-scale simulation rather than extensive real data collection, this approach enables the generation of diverse object-interaction data, improving accuracy and adaptability while reducing cost and risk, with applications in hazardous material handling, laboratory automation, and remote scientific operations.
Competition history
- ISEF 2026
Resources
Related projects
ISEF · 2025
Low-Cost, High-Performance Humanoid Upper Body for Automating Hazardous Tasks With Quasi-Direct Drive Actuators and Machine Learning
ISEF · 2026
Effectiveness of AI-Adaptive Robotic Hand in Common Medical and Industrial Tasks
ISEF · 2026
Alternative Distillation Methods in Sim-to-Real Transfer
CWSF · 2026
From Single-Task Imitation to Real-World Generalization: A Hierarchical Reason-Act Robot System
Closest projects by meaning, across every fair and year in the corpus.
Source: Regeneron International Science and Engineering Fair