Hand gesture and voice-controlled mouse for physically challenged using computer vision
Keywords:
HCI, Gesture, AI, Media pipe, Virtual MouseAbstract
A Human-Computer Interface (HCI) is presented in this paper to allow users to control the mouse cursor with hand gestures and voice commands. The system uses computer vision EfficientNet B4 architecture with no code ml to identify different hand gestures and map them to corresponding cursor movements. The objective is to create a more efficient and intuitive way of interacting with the system. The primary purpose is to provide a reliable and cost-effective alternative to existing mouse control systems, allowing users to control the mouse cursor with hand gestures and voice commands. The system is designed to be both intuitive and user-friendly, with a simple setup process. The highly configurable system allows users to customize how it works to suit their needs best. The system's performance is evaluated through several experiments, which demonstrate that the hand gesture-based mouse control system can accurately and reliably move the mouse cursor. Overall, this system can potentially improve the quality of life and increase the independence of individuals with physical disabilities.
References
Pavate, A., Mistry, J., Palve, R., & Gami, N. (2020). Diabetic Retinopathy Detection-MobileNet Binary Classifier.
Pavate, A., & Ansari, N. (2015). Risk Prediction of Disease Complications in Type 2 Diabetes Patients Using Soft Computing Techniques. 2015 Fifth International Conference on Advances in Computing and Communications (ICACC), 371-375.
Kumar, A., Pavate, A., Abhishek, K., Thakare, A. R., & Shah, M. (2020). Landmines Detection Using Migration and Selection Algorithm on Ground Penetrating Radar Images. 2020 International Conference on Convergence to Digital World - Quo Vadis (ICCDW), 1-6.
Pavate, A., & Bansode, R. S. (2020). Performance Evaluation of Adversarial Examples on Deep Neural Network Architectures.
Pavate, A., & Bansode, R. (2023). Design and Analysis of Adversarial Samples in Safety–Critical Environment: Disease Prediction System. In: Gupta, M., Ghatak, S., Gupta, A., Mukherjee, A.L. (eds) Artificial Intelligence on Medical Data. Lecture Notes in Computational Vision and Biomechanics, vol 37. Springer, Singapore.
Shi, B., Hsu, W., Lakhotia, K., & Mohamed, A. (2022). Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction. ArXiv, abs/2201.02184.
K. H. Shibly, S. Kumar Dey, M. A. Islam and S. Iftekhar Showrav, "Design and Development of Hand Gesture Based Virtual Mouse," 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), Dhaka, Bangladesh, 2019, pp. 1-5. https:doi.org/10.1109/ICASERT.2019.8934612.
Titlee, R., Rahman, A. U., Zaman, H. U., & Rahman, H. A. (2017). A novel design of an intangible hand gesture controlled computer mouse using vision based image processing. In 2017 3rd International Conference on Electrical Information and Communication Technology (EICT) (pp. 1-4). Khulna, Bangladesh.
Varun, K. S., Puneeth, I., & Jacob, T. P. (2019). Virtual Mouse Implementation using Open CV. In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI) (pp. 435-438). Tirunelveli, India.
Reddy, V. V., Dhyanchand, T., Krishna, G. V., & Maheshwaram, S. (2020). Virtual Mouse Control Using Colored Finger Tips and Hand Gesture Recognition. In 2020 IEEE-HYDCON, Hyderabad, India (pp. 1-5).
Chowdhury, S. R., Pathak, S., & Praveena, M. D. A. (2020). Gesture recognition based virtual mouse and keyboard. In 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI)(48184) (pp. 585-589). Tirunelveli, India.
Sharma, Neeta & Gupta, Aviral. (2020). A Real Time Air Mouse Using Video Processing. International Journal of Advanced Science and Technology, 29, 4635 - 4646.
Mishra, P., & Sarawadekar, K. (2019, December). Fingertips detection in egocentric video frames using deep neural networks. In 2019 International Conference on Image and Vision Computing New Zealand (IVCNZ) (pp. 1-6). IEEE.
