A Computer Vision Based Human Computer Interaction System Using Nose Movement for Assistive Control

Overview

A Computer Vision Based Human Computer Interaction System Using Nose Movement for Assistive Control Abstract: Many people with physical disabilities have difficulty using computers and games because most systems require hand movement. This project presents a hands-free computer control system that uses computer vision to track nose movement through a webcam. The detected nose movement is used to control a character in a simple game, which serves as a way to test the system’s responsiveness. The project was created using web-based technologies and does not require any special hardware. The results show that nose movement can be used as a reliable input method, demonstrating how computer vision can improve accessibility and human–computer interaction. Background Research: This project builds on these ideas by applying nose tracking to a simple game-based system, demonstrating how computer vision can be used to create an affordable and accessible hands-free interaction method without specialized equipment. Problem Statement: Most computer games and applications require a keyboard, mouse, or controller, which are difficult or impossible to use for people with limited motor control. As a result, many physically disabled individuals are unable to access digital entertainment and interactive systems. There is a need for a simple and affordable hands-free control method that allows users to interact with computers without using their hands. Hypothesis: If nose movement is detected and tracked through computer vision, then hands-free control of a computer game is possible, which can enhance accessibility for users with limited motor abilities. Objectives: • To create a hands-free game controlled by nose movement • To use computer vision to track facial movement • To demonstrate how AI can be used for assistive technology • To design a low-cost system that works on a regular computer   Methodology: • A webcam captures live video of the user • Computer vision is used to detect the user’s nose • The nose position is tracked in real time • Movement of the nose controls the character in the game Technologies Used • HTML • CSS • JavaScript, Java, Python, Github • Computer Vision • Webcam Procedural Steps: 1. Set up a computer with a webcam and ensure proper lighting. 2. Create a simple browser-based game using HTML, CSS, JavaScript, Java, and Python. 3. Integrate a computer vision library to detect the user’s face. 4. Identify and track the nose position in real time. 5. Record nose movement coordinates from the webcam feed. 6. Map left and right nose movement to character movement in the game. 7. Test the system under different lighting conditions and distances. 8. Observe responsiveness and accuracy of the nose-controlled input. 9. Record results and note any limitations or errors. 10. Analyze whether the system allows effective hands-free control. 11. Once done, deploy the code into GitHub and make it a repository for the public. 12. Test it out three times with 9 people, using the deployed link, and make sure it works properly. Applications: • Accessible gaming for physically disabled users • Assistive computer interaction • Rehabilitation engagement activities • Demonstration of inclusive technology design Limitations: • Webcam quality affects accuracy • No testing in medical or hospital environments Conclusion: This project shows that nose movement can be used as an effective hands-free control method using computer vision. By using simple and affordable technology, the system improves accessibility for users who cannot use traditional input devices. The project demonstrates how AI and human computer interaction can be combined to create inclusive and practical solutions. References: 1. Morikawa, C., & Lyons, M. J. (2017). Design and evaluation of vision-based head and face tracking interfaces for assistive input. arXiv preprint arXiv:1707.08019. https://arxiv.org/abs/1707.08019 2. Kabir, M. R., Polychronis, K., & Steinfeld, A. (2021). Nose tracking assistive robot control for people with motor dysfunctions. Archives of Physical Medicine and Rehabilitation, 102(10), 1968–1976. https://doi.org/10.1016/j.apmr.2021.04.016 3. Ding, G. (2023). Perspective and the use of eye tracking in human–computer interaction. Highlights in Science, Engineering and Technology, 39, 150–155. https://doi.org/10.54097/hset.v39i.6581 4. Chen, Y., & Newman, W. S. (2008). A hands-free vision-based interface for computer accessibility. Journal of Network and Computer Applications, 31(4), 357–374. https://doi.org/10.1016/j.jnca.2008.02.002

Competition history

• CSEF 2026 Mathematical Sciences (Junior Division) · Entry J-14-10

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Mathematics projects
• CSEF Mathematics projects
• Every CSEF 2026 project