System for Learning Kinetic Communication Patterns in Patients with Cerebral Palsy Based on Deep Learning

Abstract

This paper presents the design and development of a system aimed at improving communication for patients with cerebral palsy through the application of deep learning techniques. The system's components—including a video/image repository, a user interaction tool, and a kinesic language classifier—are meticulously defined and described. The system allows users to upload and evaluate videos that capture kinesic communication actions such as gestures, signs, and other forms of non-verbal communication. These actions are identified by pre-trained convolutional neural network models, providing a robust mechanism for recognizing and interpreting patient behavior. The study evaluates the performance of several deep learning models, including VGGNet, ResNet, and ResNetCRNN, within the kinesic language classifier component. Results indicate that the ResNetCRNN model outperforms the others in terms of accuracy and loss percentage, particularly due to its capability to assess sequences of related images rather than independent frames. The findings suggest that the proposed system offers a viable alternative for facilitating communication in cerebral palsy patients by analyzing and identifying their actions. The system's strength lies in its ability to learn from each patient, making it a constructive and adaptable communication tool.