Techniques for estimating the direction of pointing gestures using depth images in the presence of orientation and distance variations from the depth sensor

Abstract

A significant part of our daily life is spent interacting with various computing devices, mobile phones, etc. Soon, robots and drones may become common in daily life, as might be virtual reality based interfaces. Currently, the interaction with such devices is through the use of mouse, touch-pad, joystick, virtual reality wand, etc. Most of these interaction devices (such as mouse, keyboard and joystick) are immobile and restricted to table-top usage. Some interaction devices such as drone controllers are mobile, but need significant training before usage. Devices such as virtual reality wands are mobile, but inhibit immersive experience due to their handheld mode of operation and size of the device. Also, these interaction devices need to be touched with the hand during operation, thus making them prone to transmission of virus, bacteria etc. due to their use by multiple subjects. To overcome the limitations of the existing interaction devices, the recent trend is to move towards gesture-based user interfaces, which enable more natural interaction. Gestural inter-faces facilitate better interaction in 3D, since the user is not constrained to operate on planar surfaces (like with the use of mouse and touch-pad) or to operate at a fixed location (like joy-stick). Also, gesture based interaction requires minimum training, as human beings use gestures in day to day life. Gesture based interfaces are immersive as they mainly use the hands and not cumbersome external devices. Also, gesture based interfaces are non-contact type and minimize the probability of transmission of virus, bacteria etc. All interaction modes (gestural or device based) mentioned above are mostly used for selection tasks, pick-and-place tasks and direction indication tasks. These tasks primarily consist of pointing tasks. Existing gesture-based pointing interfaces suffer from one or more of the following limitations...