Virtual Hands: Real Time Keyboard, Desktop & Application Navigation using Gestures

ABSTRACT

In computer system keyboard is the most prominent input medium of all time. But lately, human community is living in an era of global pandemic being afraid of suffering from Coronavirus (Covid-19) and hence each and every person avoids touching anything. This is because of the fear of contracting this contagious virus and their mutants. So, to mitigate this issue, we present a method "webcam based virtual keyboard interface"to interact with a computer system. The code of this method is written using pre-built modules like OpenCV, MediaPipe, PyVDA, Win32API, etc. and Python 3.9. This approach uses matching the index finger and middle finger on the specific key. After that the virtual desktop switching mechanism is done by PyVDA. The PyttSX3 library plays the sound whenever any key is pressed or when a desktop switch is initiated, corresponding to the key pressed or the desktop switched. In this approach no additional hardware device other than the webcam, that is already available in the system, is required. This approach is also useful for those persons who wants the access the system, even when their hands are dirty. © 2022 IEEE.