Study on vision based gesture recognition for indian and american sign languages

Abstract

The world is progressing towards automation in all directions. Specially challenged people who have difficulties in verbal communication convey their views and emotions by way of simple gestures. Gestures include movement of body parts like arm, leg, head, changes in facial parts, eye blinking etc. Capturing the gestures and translating them into meaningful information is still a challenging task and it is an integral part of any human machine interaction. Sign language interpretation is one of the significant applications of gesture recognition. Some of the other applications of vision-based gesture recognition includes facial landmark detection, facial expression recognition and morphological approach to detect a human in a video scene and these are included as case studies in this thesis. The challenge in recognizing a hand or face or both depends on the problem of interest. The recognition of the region of interest becomes difficult due to face or hand pose variations, varying illumination conditions, dealing with complex backgrounds and occlusions of vital segments to name a few. The objective of this research is to develop an intelligent system that can read non-verbal communication made by human and infer information from it. The system works as a sign language interpreter that can process the captured hand gesture and make suitable interpretations pertaining to the context in which they are created. This approach can be adopted at the alphabet level or word level. The former is known as finger spelling which involves suppression of facial regions and localizing the hand areas and latter is used for mapping gestures against a predefined vocabulary thus generating a meaningful inference. There exist many sign languages across the world based on the geographical location and people s characteristic styles and custom behaviour. The Indian and American sign languages are considered in this research. The hand gestures are captured in real-time in simple and complex backgrounds and under varying lighting condi