Methodology for Continuous Sign Language Recognition with Focus on Movement Epenthesis Segmentation

Abstract

Sign language (SL) is a movement language which expresses certain semantic information through a series of hand and arm motion, facial expressions and head/body postures. The urge to support the integration of deaf people into the hearing society made the automatic sign language recognition an area of interest for the researchers. Sign Language is the basic communication medium between the deaf people. SL uses static or dynamic hand gestures, facial expressions, head/body postures, locations of hand with respect to body etc, to represent signs. The signer often uses the 3D space around his or her body to describe an event. The phonemes of sign language are included in the sign language sentences by means of facial expressions, eye gaze and head /body postures, referred to as Non Manual Signs (NMS). To decipher the full meaning of the sentence, we need to observe the NMS in the signing. Sign language recognition aims at converting the sign gestures to text or speech. Sign language involves static gestures as well as dynamic gestures, and many challenges to recognize them exist, such as the following: two handed signs with both hands moving and conveying information, signer dependent variations, large vocabulary, occlusions with hand and body, integration of the linguistic information conveyed through hands, arms, face, and head/body postures etc. The objective of this thesis is to develop and analyze efficient sign language recognition algorithms which could be used for automatically recognizing signs from the SL vocabulary. The problem of sign language recognition comprises mainly of two sub problems 1) Isolated Sign Recognition (ISR) 2) Continuous Sign Language Recognition (CSLR). Isolated sign recognition aims at taking as input (in form of a video), the dynamic gesture streams corresponding to one isolated sign from the sign language dictionary and give as output the SL word corresponding to the sign. ..