CareBotix in Motion II: Advanced Robotic Manipulation, Navigation, and Social Interaction

The population aged sixty and above is projected to grow from 1 billion in 2020 to 1.4 billion in 2030 and 2.1 billion by 2050, while OECD projections indicate a shortage of 13.5 million caregivers by 2050. Robotics and AI are emerging as essential technologies to help older adults age in place with dignity.

CareBotix II builds on the original platform. Major additions include multimodal social interaction, navigation, full‑body humanoid hardware informed by user‑driven design, and a platform‑agnostic core.

Performance metrics show high manipulation success (93–100 percent), robust localization, and reliable goal‑reach navigation, with state‑of‑the‑art conversational AI latency and emotion classification.

Iterative testing on XLeRobot, the navigation scout, and the CareBotix II platform confirmed the architecture scales with minimal modification, with platform agnostic systems identified as important for acceptance by older adults in user studies.

According to the World Health Organization, the number of adults aged 60+ will rise from 1 billion in 2020 to 2.1 billion in 2050. Concurrently, OECD projections warn of a 13.5 million person caregiver shortfall, with caregiver to older adult ratios declining from 7:1 in 2010 to less than 3:1 by 2050. This widening gap places unprecedented pressure on families, healthcare systems, and long term care infrastructure.

Canada faces similar demographic challenges. By 2052, 24.9% of Canadians could be 65 or older, with adults 85+ the fastest growing age group. Sustaining independence, mobility, and social connection for older adults will require scalable, technology supported care solutions.

Humanoid robotics is advancing, driven by breakthroughs in multimodal AI, embodied learning, and human centered design. These systems are engineered to operate in dynamic, real world environments, enabling them to support healthcare, home assistance, and daily living. Equipped with cameras, tactile sensing, and Vision-Language-Action models, humanoid robots can interpret natural language instructions, understand their surroundings, and perform multi-step actions to provide household and caregiving support.

CareBotix II is a solution in response to the caregiving crisis. Building on the original system, CareBotix II is a platform agnostic architecture integrating navigation, manipulation, social interaction and multimodal AI. CareBotix II performs object handling tasks, interprets navigation goals, and engages older adults through expressive, context aware interaction, demonstrating how next generation humanoid robots can help close the caregiving gap and support older adults to age with dignity, independence, and safety.

CareBotix II Platform Design:

CareBotix II is a modular, full stack humanoid system that aims to integrate manipulation, navigation, and social interaction. The robot combines XLeRobot derived arms, a precision mobile base, and head hardware into a lightweight frame built from 3D printed joints and aluminum extrusions. A dual processor design separates responsibilities: Jetson Orin Nano manages social interaction, while the Jetson Orin NX executes navigation, manipulation, and sensor fusion. Multimodal sensing (including RGB cameras, 4D LiDAR, IMU, gesture cameras, and directional audio) provides the robot with an understanding of its environment and user behavior.

Manipulation, VLA Reasoning, and Navigation:

Manipulation is achieved through a platform agnostic control stack that combines Vision Language Action (VLA) and Action Chunking Transformer (ACT) with imitation learning and level reasoning. Demonstrations collected through the LeRobot pipeline train ACT models to produce motion primitives for tasks such as egg handling, cleaning, and pill sorting. Task interpretation is performed by Isaac GR00T VLA, which integrates visual perception, language understanding, and action generation. Navigation uses a ROS 2 pipeline that fuses RGB D, IMU, and LiDAR data into a point cloud based map. A perception layer applies people segmentation and YOLOv8 to remove humans and static objects. Cosmos Reason 2B generates semantic navigation instructions, which Nav2 will ultimately convert into smooth, obstacle aware motion.

Social Interaction and Human Centered Iterative Design:

The social subsystem integrates full body pose estimation, gesture recognition, facial expression rendering, and conversational AI. A custom Multi MiniLM pipeline classifies user inputs for toxicity, sentiment, intent, and emotion, enabling safe, empathetic, and context aware responses. Development followed an iterative, platform agnostic approach: XLerobot validated manipulation, The Scout refined navigation, and CareBotix II aims to integrate all subsystems. Human centered design included structured sessions with older adults, impacting the robot’s design, behaviour, and interactions.

The Full System Architecture:

The full system architecture begins with OpenWakeWord and Whisper on the Jetson Orin Nano, which classify the user’s speech as either a social interaction or a task command.

If the user says “Hello CareBotix”, the input is routed to the head subsystem, where the RGB camera, microphone array, MiniLM, Gemma, Piper TTS, the expressive face renderer, and the 3‑DoF head controller generate natural social interaction. If the user says “Hello CareBotix Task”, the command is sent to the Jetson Orin NX, which runs the full‑body ROS autonomy stack. Wake word accuracy trained to close to 100%

Manipulation:

The Isaac GR00T N1.5 architecture uses a Vision-Language-Action (VLA) system for high level reasoning. It interprets natural language goals and visual context, identifies objects, and produces step wise sub-actions that describe what the robot should do.

In addition, Action Chunking Transformer (ACT) models (trained through LeRobot) were used and evaluated. ACT learns manipulation skills directly from demonstrations and produces smooth, reusable action chunks that generalize across tasks.

Demonstrations included picking up an egg, a bimanual cleaning task, and a health focused task – placing and organizing pills in a basket. The egg pick and place achieved an accuracy between 97-100%. The Bimanual cleaning task achieved an accuracy of between 98-100% and the pill and bottle placing and organizing task achieved an accuracy of between 93-97%.

Social Intelligence:

The multimodal social‑interaction system combines gesture recognition, expressive rendering, directional audio, and conversational intelligence. The robot uses real‑time pose estimation to identify hand shapes, arm movements, and full‑body posture. A facial display engine generates dynamic expressions, emotion cues, and accurate lip‑syncing to support natural, human‑like interaction.

The latency of my MiniLm/Deberta Model was 0.1-0.3 seconds on the Jetson Orin Nano, which is 0.2-0.7 seconds better than the current SOTA in latency (DistilBERT) at 0.3-1.0 seconds on the Jetson Orin Nano.

For human–robot interaction, a custom fine-tuned Multi-MiniLM architecture used four classifiers (to label text): Toxicity, Sentiment, Intent, and Emotion. My Multi-MiniLM model achieved SOTA for emotion classification (GoEmotions, 27 class) achieving an accuracy of 82%, greater than SOTA models that are deployable on the Jetson Orin Nano – with a SOTA of 60-65%.

Navigation and Systems Integration:

The system uses a Gemini Orbbec / Intel RealSense sensor suite, combining RGB, Depth, and IMU data. Each stream is published as ROS 2 topics, and the mapping nodes subscribe to them to build a point‑cloud‑based representation of the environment.

Once the map is created, a perception layer applies people segmentation and YOLOv8 to filter out humans and label static objects. This annotated map is then passed to Cosmos Reason 2B, which uses a visual encoder and language‑based reasoning to generate high‑level semantic instructions.

Isaac ROS VSLAM initialized maps in 10 s, maintained 5–10 cm accuracy, and tracked 800–1200 features per frame. Cosmos Reason 2B achieved >90% goal‑reach performance on obstacle‑aware navigation.

The Design:

CareBotix II is a proprietary design driven by older adult needs and preferences. The robot is a bimanual, height adjustable, environment aware, conversational robot, operating on disconnected, edge devices. The development of the agnostic platform can enable scalable, cross-platform deployment, ensuring equitable access and continuity of care for older adults.

Sub-System Level Performance:

CareBotix II achieved 93–100% manipulation success, >90% navigation goal reach, SOTA for conversational AI latency and emotion classification. Stable grasp trajectories, 1–2 cm pose error, SLAM initialization and feature tracking, confirm the perception to action pipeline is accurate, resilient to noise, and capable of real time operation on embedded hardware.

Integrated Reasoning and Control:

Vision Language Action reasoning, semantic mapping, and Cosmos based planning enabled CareBotix II to interpret natural language instructions and execute multi step tasks. Manipulation routines transferred across robotic platforms, validating the platform agnostic architecture. Navigation fused SLAM, semantic labeling, and reasoning to convert user commands into actionable waypoints. The social subsystem maintained conversational context, produced natural expressions, with timely response - mimicking human conversations.

Impact and Real World Significance:

Human subject evaluation showed 96–98% trust and 96% positive ratings for facial and gesture systems. Participants described the robot as predictable, expressive, and emotionally supportive - qualities essential for senior care. These findings highlight the potential for humanoid robots to reduce caregiver burden, support aging in place, and deliver long term cost savings. CareBotix II advances socially intelligent humanoids capable of assisting in homes, clinics, and long term care settings.

Future development will expand CareBotix II toward full humanoid capability and improved real world performance. First is integrating bipedal legs - enabling stair climbing, uneven surface traversal, and body mobility. Next, lighter, higher torque actuators will improve strength to weight, manipulation, and dynamic tasks. The platform agnostic architecture will evolve into a modular front end, allowing perception, control, and reasoning across multiple robotic bodies. Finally, advancements in Vision-Language-Action reasoning and inverse kinematics will enable more complex, multi-step tasks with grounding between language and physical action, moving CareBotix II closer to a deployable humanoid assistant for care environments.

I would like to begin by acknowledging and thanking my teachers and parents. My teachers’ support for STEM has enabled student access to these wonderful regional and national programs. Thank you to my parents for their caregiving over many long nights spent preparing for BASEF and CWSF.

My heartfelt appreciation to our BASEF mentors. Their constant guidance enabled me to achieve even more in terms of the report, board, interview, and overall preparation for CWSF.

Finally, I would like to acknowledge the generosity and continuous support of my sponsors and the organizers of CWSF.  The opportunities that students receive would not have been possible without your kindness. I sincerely thank you for all that you do for students across Canada. These memories have positively impacted me, building perseverance, resilience, and guiding me on my career path forward in robotics and AI.

1) Coherent Market Insights. (2026, March). The personal and caregiving segment is expected to capture 36% share of the market in 2026. Retrieved from https://www.coherentmarketinsights.com/industry-reports/humanoid-robot-market

2) Fadelli, I. (2025a, November 27). Humanoid robots reliably manipulate different objects with 87% success using new framework. Tech Xplore. Retrieved from https://techxplore.com/news/2025-11-humanoid-robots-reliably-success-framework.html

3) Fadelli, I. (2025b, December 17). Robotic arm successfully learns 1,000 manipulation tasks in one day. Tech Xplore. Retrieved from https://techxplore.com/news/2025-12-robotic-arm-successfully-tasks-day.html

4) Lee, S. H., Kim, J. S., & Yu, S. (2025). The impact of care robots on older adults: A systematic review. Geriatric Nursing, 65, 103507. https://doi.org/10.1016/j.gerinurse.2025.103507

5) Market.us. (2025, March). Global humanoid robot market. Retrieved from https://market.us/report/humanoid-robot-market/

6) National Institute on Ageing. (2022). Ageing in the right place: Supporting older Canadians to live where they want. Retrieved from https://static1.squarespace.com/static/5c2fa7b03917eed9b5a436d8/t/638e0857c959d1546d9f6f3a/1670252637242/AIRP+Report+Final2022-.pdf

7) NVIDIA. (2025). NVIDIA Isaac GR00T: NVIDIA Isaac GR00T N1 is the world’s first open foundation model for generalized humanoid robot reasoning and skills [Computer software]. GitHub. Retrieved from https://github.com/NVIDIA/Isaac-GR00T

8) OECD. (2020). Who cares? Attracting and retaining care workers for the elderly. Retrieved from https://www.oecd.org/content/dam/oecd/en/publications/reports/2020/05/who-cares-attracting-and-retaining-care-workers-for-the-elderly_735ab920/92c0ef68-en.pdf

9) Ribeiro, O., Araújo, L., Figueiredo, D., Paúl, C., & Teixeira, L. (2021). The caregiver support ratio in Europe: Estimating the future of potentially (un)available caregivers. Healthcare, 10(1), 11. https://doi.org/10.3390/healthcare10010011

10) Song, Y., & Chung, K. (2025). Facial and speech-based emotion recognition using sequential pattern mining. Electronics, 14(20), 4015. https://doi.org/10.3390/electronics14204015

11) Statistics Canada. (2017, May). A portrait of the population aged 85 and older in 2016 in Canada. Retrieved from https://www12.statcan.gc.ca/census-recensement/2016/as-sa/98-200-x/2016004/98-200-x2016004-eng.cfm

12) Wikimedia Commons. (2008, June 23). File: Inverse-kinematics-multiple-solutions.svg. Retrieved from https://commons.wikimedia.org/wiki/File:Inverse-kinematics-multiple-solutions.svg

13) World Health Organization. (2025, October 1). Ageing and health. Retrieved from https://www.who.int/news-room/fact-sheets/detail/ageing-and-health

14) Wu, Z., Zhou, Y., Xu, X., Wang, Z., & Yan, H. (2025). MoManipVLA: Transferring vision-language-action models for general mobile manipulation. arXiv. Retrieved from https://arxiv.org/abs/2503.13446

15) Zhang, J., Chen, Y., Xu, Y., Huang, Z., Zhou, Y., Yuan, Y.-J., Cai, X., Huang, G., Quan, X., Xu, H., & Zhang, L. (2025). 4D-VLA: Spatiotemporal vision-language-action pretraining with cross-scene calibration. arXiv. Retrieved from https://arxiv.org/abs/2506.22242