Hand gesture and voice-controlled mouse for physically challenged using computer vision
Main Article Content
Abstract
A Human-Computer Interface (HCI) is presented in this paper to allow users to control the mouse cursor with hand gestures and voice commands. The system uses computer vision Efficient Net B4 architecture with no code ml to identify different hand gestures and map them to corresponding cursor movements. The objective is to create a more efficient and intuitive way of interacting with the system. The primary purpose is to provide a reliable and cost-effective alternative to existing mouse control systems, allowing users to control the mouse cursor with hand gestures and voice commands. The system is designed to be both intuitive and user-friendly, with a simple setup process. The highly configurable system allows users to customize how it works to suit their needs best. The system's performance is evaluated through several experiments, which demonstrate that the hand gesture-based mouse control system can accurately 100% and reliably move the mouse cursor. Overall, this system can potentially improve the quality of life and increase the independence of individuals with physical disabilities.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
Pavate, A., Mistry, J., Palve, R., & Gami, N. (2020). Diabetic retinopathy detection-MobileNet binary classifier. Acta Scientific Medical Sciences, 4(12), 86-91.
Pavate, A., & Ansari, N. (2015, September). Risk prediction of disease complications in type 2 diabetes patients using soft computing techniques. In 2015 Fifth International Conference on Advances in Computing and Communications (ICACC) (pp. 371-375). IEEE.
Kumar, A., Pavate, A., Abhishek, K., Thakare, A., & Shah, M. (2020, February). Landmines detection using migration and selection algorithm on ground penetrating radar images. In 2020 International Conference on Convergence to Digital World-Quo Vadis (ICCDW) (pp. 1-6). IEEE.
Pavate, A. A., & Bansode, R. (2020). Performance evaluation of adversarial examples on deep neural network architectures. In Intelligent Computing and Networking: Proceedings of IC-ICN 2020 (pp. 239-251). Singapore: Springer Singapore.
Pavate, A., & Bansode, R. (2022, July). Design and analysis of adversarial samples in safety–critical environment: Disease prediction system. In Artificial Intelligence on Medical Data: Proceedings of International Symposium, ISCMM 2021 (pp. 349-361). Singapore: Springer Nature Singapore.
Pavate, A. A., & Bansode, R. (2022). Evolutionary algorithm with self-learning strategy for generation of adversarial samples. International Journal of Ambient Computing and Intelligence (IJACI), 13(1), 1-21.
Hamal, S. N. G., Ulvi, A., Yiğit, A. Y., & Yakar, M. (2022). Su altı yapılarının 3B modellemesi ve dokümantasyonunda kullanılan video ve fotoğraf çekimi yöntemlerinin karşılaştırmalı analizi. Journal of the Institute of Science and Technology, 12(4), 2262-2275.
Lalitha, R. V. S. S., & Srinivasu, P. N. (2017). An efficient data encryption through image via prime order symmetric key and bit shuffle technique. In Computer Communication, Networking and Internet Security: Proceedings of IC3T 2016 (pp. 261-270). Springer Singapore.
Srinivasu, P. N., Norwawi, N., Amiripalli, S. S., & Deepalakshmi, P. (2021). Secured compression for 2D medical images through the manifold and fuzzy trapezoidal correlation function. Gazi University Journal of Science, 35(4), 1372 – 1391
Yakar, M., Ulvi, A., & Toprak, A. S. (2015). The problems and solution offers, faced during the 3D modeling process of Sekiliyurt underground shelters with terrestrial laser scanning method. International Journal of Environment and Geoinformatics, 2(2), 39-45.
Shi, B., Hsu, W. N., Lakhotia, K., & Mohamed, A. (2022). Learning audio-visual speech representation by masked multimodal cluster prediction. arXiv preprint arXiv:2201.02184.
Shibly, K. H., Dey, S. K., Islam, M. A., & Showrav, S. I. (2019, May). Design and development of hand gesture based virtual mouse. In 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT) (pp. 1-5). IEEE.
Titlee, R., Rahman, A. U., Zaman, H. U., & Rahman, H. A. (2017, December). A novel design of an intangible hand gesture controlled computer mouse using vision based image processing. In 2017 3rd International Conference on Electrical Information and Communication Technology (EICT) (pp. 1-4). IEEE.
Varun, K. S., Puneeth, I., & Jacob, T. P. (2019, April). Virtual mouse implementation using open CV. In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI) (pp. 435-438). IEEE.
Reddy, V. V., Dhyanchand, T., Krishna, G. V., & Maheshwaram, S. (2020, September). Virtual mouse control using colored finger tips and hand gesture recognition. In 2020 IEEE-HYDCON (pp. 1-5). IEEE.
Chowdhury, S. R., Pathak, S., & Praveena, M. A. (2020, June). Gesture recognition based virtual mouse and keyboard. In 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI) (48184) (pp. 585-589). IEEE.
Mishra, P., & Sarawadekar, K. (2019, December). Fingertips detection in egocentric video frames using deep neural networks. In 2019 International Conference on Image and Vision Computing New Zealand (IVCNZ) (pp. 1-6). IEEE.
Gupta, A., & Sharma, N. (2020). A real time air mouse using video processing. International Journal of Advanced Science and Technology. 29, 4635-4646.
Chaurasia, R., Tiwari, A., Vishwakarma, A., Mandal, B., & Agnihotri, U. (2020). Hand gesture recognition and voice conversion system. International Journal of Creative Research Thoughts (IJCRT), Uttar Pradesh, 8(7), 1993-1997
Manisha, K., & Venkataramana, T. (2020). Arduino based gestures to speech converter gloves for deaf and dumb people. Journal of Emerging Technologies and Innovative Research (JETIR), 10(5), 220-223
Yasen, M., & Jusoh, S. (2019). A systematic review on hand gesture recognition techniques, challenges and applications. PeerJ Computer Science, 5, e218.
Sultana, A., Ahmed, F., & Alam, M. S. (2022). A systematic review on surface electromyography-based classification system for identifying hand and finger movements. Healthcare Analytics, 100126.
Ibraheem, N., & Khan, R. Z. (2012). Hand gesture recognition: A literature review. International Journal of Artificial Intelligence & Applications (IJAIA), 3, 161-174.
Patel, K., Solaunde, S., Bhong, S., & Pansare, S. (2022). Virtual mouse using hand gesture and voice assistant. Internatıonal Journal of Innovatıve Research ın technology, 9(2), 84-88
Huang, H., Chong, Y., Nie, C., & Pan, S. (2019, June). Hand gesture recognition with skin detection and deep learning method. In Journal of Physics: Conference Series (Vol. 1213, No. 2, p. 022001). IOP Publishing.
Morajkar, A., James, A. M., Bagwe, M., James, A. S., & Pavate, A. (2023). Hand gesture and voice-controlled mouse for physically challenged using computer vision. Advanced Engineering Days (AED), 6, 127-131.