Dynamic Indian Sign Language Character Recognition: Using HOG Descriptor and a Customized Neural Network

Abstract

Communication is the exchange of thoughts, messages, or information, by speech, visual signals, writing, or behaviour. Deaf and dumb people communicate among themselves using sign languages, but they find it difficult to expose themselves to the outside world. This paper proposes a method for recognizing Indian sign language (ISL) Two hand characters given as input by the user in the form of hand gestures. There are 18 ISL characters which are shown using two hands. Unlike the conventional method, this method does not require any additional hardware and makes the user comfortable. The system takes input at real time through a webcam integrated in the laptop. The hand region is separated out from the background using skin segmentation and motion segmentation. Since the segregation of overlapping hands is difficult, two hand characters are identified using HOG features. A back propagation neural network is used as the learning algorithm to make the system adaptable for different users. The system has been tested for several people of varying skin complexions, in several environments and was found to have accuracy of about 90%. The accuracy mainly dropped due to the illumination of the environment and occlusion of hands involved in two hand gestures.