Compact Thai Sign Language Translation by Deep Learning

Abstract

Sign language translation is a challenging problem in natural language processing. Its principle involves machine translation from sign language images to spoken language text. Designing a good translation is not a trivial task since there are a large number of both input image pixels and output classes. We propose the deep learning model to translate static gestures of Thai sign language (TSL) to the corresponding Thai spoken words. The main objective is to design a compact model that delivers high performance so that it can be implemented on mobile devices. Several mobile convolutional neural networks (CNN) are investigated to find the best backbone architecture. We also attach additional layers to the selected CNN architecture to fine-tune its performance. The experiments on the dataset collected from twenty-four volunteers indicate excellent results; in terms of precision, recall, and f1-score, of the proposed model. The comparisons with the state-of-the-art models and the feature visualizations from convolution layers endorse its effectiveness.