Knowledge Technology Corpora

Gestures constitute a crucial element in human communication, as well as in human-robot interaction, thus, gesture recognition has been a field of particular interest in computer science. More specifically, dynamic gesture recognition is a challenging task, since it requires the accurate detection of the body parts involved in the gesture, their tracking and the interpretation of their sequential movement.

To evaluate our systems on a dynamic command gesture recognition task, we designed, recorded and organized the TsironiGR corpus. This corpus contains nine command gestures for Human-Robot Interaction (HRI): Abort, Circle, Hello, No, Stop, Turn Right, Turn Left, and Warn. Six different subjects were recorded, each one performing at least ten times the same gesture. We recorded a total of 543 sequences. Each of the gesture sequences is segmented and labeled with one of the nine classes. The gestures were captured by an RGB camera with a resolution of 640x480, recorded with a frame rate of 30 FPS.

The gesture descriptions are provided below:

• Abort - 57 Sequences -It is a gesture that requires the motion of the hand in front of the throat with the palm facing downwards.
• Circle - 60 Sequences - This gesture is a cyclic movement starting from the shoulder with the arm and the index finger being stretched out. The movement is taking place in front of the body of the user while he/she is facing the direction of the capturing sensor.
• Hello - 59 Sequences - As its name denotes, it is the typical greeting gesture, with the hand waving while facing the direction of the robot. The waving starts from the elbow rotation.
• No - 62 Sequences - To perform this gesture the arm and the index finger need to be stretched out towards the direction of the robot and then the repeating wrist rotation alternately to the right and the left.
• Turn Left - 62 Sequences - As the name of gesture denotes, the arm is pointing to the left.
• Turn Right - 60 Sequences - In the same sense as Turn Left, this gesture consists of the arm pointing to the right.
• Stop - 60 Sequences - The gesture includes raising the arm in front of the body with the palm facing the robot.
• Turn - 63 Sequences - This gesture is a cyclic movement starting with a rotation from the elbow including the simultaneous rotation of the wrist with the index finger stretched pointing downwards. The signature of this movement seems like a circle.
• Warn - 60 Sequences - This gesture firstly requires raising the hand in front of the body having an angle between the upper and the lower arm greater than 90 degrees and then rotating the elbow while the palm is being stretched out.

License:

This corpus is distributed under the Creative Commons CC BY-NC-SA 3.0 DE license. If you use this corpus, you have to agree with the following itens:

• To cite our reference in any of your papers that make any use of the database. The references are provided at the end of this page.
• To use the corpus for research purpose only.
• To not provide the corpus to any second parties.

Contact:

To have access to this corpus, please send an e-mail with your name, affiliation and research summary to: Pablo Barros

Reference:

Tsironi, E., Barros, P., and Wermter, S. Gesture Recognition with a Convolutional Long Short-Term Memory Recurrent Neural Network. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), pp. 213-218, Bruges, Belgium, 2016.