1. UKM Learning and Research Repository
2. PHD Thesis / Tesis PHD
3. Faculty of Information Science and Technology / Fakulti Teknologi dan Sains Maklumat

Title:	Hand gesture recognition based on isolated and continuous sign language
Authors:	Ahmad Yahya Dawod (P73056)
Supervisor:	Md. Jan Nordin, Assoc. Prof. Dr.
Keywords:	Gesture -- Data processing
Issue Date:	7-Apr-2018
Description:	Hand gesture recognition from continuous sign language has attracted research interest in computer vision and human-computer interaction. The need for real-time recognition of continuous sign language has grown with the emergence of better-capturing devices such as Kinect sensors. However, the recognition of sign from the continuous input is not an easy task. One has to accurately segment the hand and fingers, track them, segment individual gestures through movement epenthesis recognition, and find the best method for classification in order to recognize the gestures. Existing approaches are mostly limited to the optical cameras that produce the low quality of input image. In addition, due to the difficult nature of the tasks involved, most works only focused on recognizing static and isolated signs rather than dynamic and continuous one. In sign language domain, very few reported work that attempts to propose a complete solution that can detect static, isolated, dynamic and continuous gestures all at the same time. This thesis presents real-time hand gesture detection and recognition from continuous sign language. The two categories in sign language recognition, namely the static gestures and the dynamic gestures were explored. For static gestures, a new technique for hand and fingertips detection using two different approaches based on YCbCr color space and skeletonization were presented respectively. In the first approach, adaptive skin color model in YCbCr colorspace that models the skin color was proposed. This is different from the normal approach that allocates specific range to model skin color which proves to be robust to varying background and lighting condition. In the second approach, skeletonization that is followed by gradient-based circle detection method were proposed to detect fingertips and track them. In dynamic gesture category, a new technique is proposed that employs contrast adjustment and gesture detection analysis to determine the start and end points of each individual moving gestures from a series of continuous gestures. This is also known as a process to detect the movement epenthesis. The technique is able to detect and identify the movement epenthesis with reasonable accuracy. To complete the research work, an approach has been developed that can recognize static and dynamic gestures in real time using the Kinect sensor. The contribution in the developed approach involves tracking the 3D points (X, Y, Z) of the hand centroid as features. Related works only use 2D points (X, Y). To test the effectiveness of the proposed technique, a new dataset set was developed. This new dataset is different from the existing dataset because it includes both signs performed for positive and negative sentences which have not been addressed in other dataset. For the classification and recognition steps of the gestures, four main tools were employed which include Support Vector Machine (SVM), Hidden Condition Random Field (HCRF), Hidden Markov Model (HMM), and Random Decision Forest (RDF). The proposed approach was tested for gestures that involve one and two hands and was compared with other approach and gave better accuracy. The recognition accuracy of the movement and orientation of the hands of the proposed technique is higher than 99.1%. The performance of the result is also compared with other two datasets,and the proposed method is able to obtain recognition rate that is up to 99.9%.,Certification on Master's/Doctoral Thesis" is not available
Pages:	292
Call Number:	TK7882.P3D339 2018 3 tesis
Publisher:	UKM, Bangi
Appears in Collections:	Faculty of Information Science and Technology / Fakulti Teknologi dan Sains Maklumat

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.