Ricerc@Sapienza

Hand gesture recognition is still a topic of great interest for the computer vision community. In particular, sign language is a foremost area of interest due to their importance in Human-Human Communication (HHC). The sign language derives from the composition of the movements of the hand (and his fingers) with respect to the pose of the body (precisely the upper part and the entire region of the torso) of the person performing the gesture.
Any hand movement can be represented by sets of feature vectors that change over time. Recurrent Neural Networks (RNNs) are suited to analyse this type of sets thanks to their ability to model the long term contextual information of temporal sequences.
Instead, the Convolutional Neural Networks (CNNs) are suitable to learn the relevant features able to discern the various poses taken by the person while performing a complex gesture.
For this reasons, in this project, we will go to design an innovative network based on a combination of CNN and LSTM. This network will be trained by using as features the angles formed by the finger bones of the human hands and a video sequence of the movements of the upper part of the bust of the human body. A further work required for this project concerns the creation of a very huge dataset containing the data of the gestures, based on the Italian Sign Language (LIS), performed by a large number of people. Acquiring data from many people will capture the different nuances between the gestures performed by different people.