Data Glove-Based Sign Language Translation with Convolutional Neural Networks

Abstract

This research was carried out because of the communication barriers that currently exist between hearing impaired and hearing people. These barriers hinder their integration into society and affect their interpersonal relationships. The objective of the study was to propose the development of a stationary assistive robot capable of displaying sign language

interpretation through the combination of data gloves and the D- CNN and LSTM algorithm to facilitate the communication of

hearing-impaired children in Huancayo. The triple diamond research design was used, where the mind map and the lotus diagram were used for the delimitation and definition of the problem. In addition, the IDEF0 technique was used to obtain a structured design of the project system. A morphological matrix was also used to choose the best solution for the problem. The chosen design contemplates the use of an Arduino UNO, flex sensors, accelerometers and gyroscopes for sign detection. The main algorithm consists of the union of a deep convolutional neural network and a LSTM for a correct sign classification module. The proposed design proposes to visualize the conceptual development of the project mentioned above.