Human Behavior Analysis

Human Behavior is an interplay of affective, cognitive, and behavioral responses of humans, and understanding human behavior is essential for many tasks of human computer interaction.
We are especially interested in tasks such as gesture recognition, in particular sign language recognition, emotion/affect recognition, and personality recognition. We have also worked on related tasks such as voice activity recognition, person detection and tracking, or simulating the human visual attention system.

We organized a Workshop on Human Behavior Analysis (Feb. 3rd, 2022)
Noha Sarhan won the Helmholtz AI PhD Pitch Award for her work on "Sign Language Recognition using 3D Convolutional Neural Networks". Please enjoy the video here.

Our publications in the field of human behavior analysis:

Noha Sarhan, Simone Frintrop: Sign, Attend and Tell: Spatial Attention for Sign Language Recognition, IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2021
Noha Sarhan, Mikko Lauri, Simone Frintrop: Multi-Phase Fine-Tuning: A New Fine-Tuning Approach for Sign Language Recognition, accepted for the German Journal of Artificial Intelligence (KI - Künstliche Intelligenz)
Stefanie Stoppel, Simone Frintrop: Investigating Bias in Facial Expression Recognition, CVPR Workshop Responsible Computer Vision, 2021, [PDF], [Poster]
Danu Caus, Guillaume Carbajal, Timo Gerkmann, Simone Frintrop: See the silence: improving visual-only voice activity detection by optical flow and RGB fusion, International Conference on Computer Vision Systems (ICVS) 2021
Noha Sarhan, Simone Frintrop: Transfer Learning of Videos: From Action Recognition to Sign Language Recognition, International Conference on Image Processing, 2020 (ICIP ’20), [PDF]
Simone Frintrop: Computational Visual Attention, A. A. Salah and T. Gevers, editors, Computer Analysis of Human Behavior, Advances in Pattern Recognition. Springer, 2011
Simone Frintrop, Erich Rome and Henrik I. Christensen: Computational Visual Attention Systems and their Cognitive Foundation: A Survey, ACM Transactions on Applied Perception (TAP), Vol. 7, Issue 1, 2010 [PDF]
An overview of the work was presented as invited talk at the Symposium on Applied Perception in Graphics and Visualization (APGV), August 27th, 2011 [Slides of talk (pdf)]
Simone Frintrop, Achim Königs, Frank Hoeller and Dirk Schulz: A Component-based Approach to Visual Person Tracking from a Mobile Platform, International Journal of Social Robotics, Special Issue on People Detection and Tracking, Vol. 2, Issue 1, pp. 53-62, Springer Netherlands, 2010 [PDF]
Simone Frintrop, Achim Königs, Frank Hoeller and Dirk Schulz: Visual Person Tracking Using a Cognitive Observation Model, Workshop on People Detection and Tracking at the IEEE 2009 International Conference Robotics and Automation (ICRA '09), Kobe, Japan, May 2009 [PDF]
Simone Frintrop and Markus Kessel: Cognitive Data Association for Visual Person Tracking, Proc. of the 1st IEEE Workshop on Human Detection from Mobile Platforms (HDMP '08) , Pasadena, California, May 2008 [PDF]
Simone Frintrop, Maria Klodt, Erich Rome: A Real-time Visual Attention System Using Integral Images, in Proc. of the 5th International Conference on Computer Vision Systems (ICVS 2007), Bielefeld, Germany, March 2007 [PDF]
Simone Frintrop: VOCUS: A Visual Attention System for Object Detection and Goal-directed Search, Phd thesis, Lecture Notes in Artificial Intelligence (LNAI), Vol. 3899 / 2006, Springer Berlin/Heidelberg. ISBN: 3-540-32759-2 [PDF], [©Springer], [see also here]
Simone Frintrop, Gerriet Backer, Erich Rome: Goal-directed Search with a Top-down Modulated Computational Attention System, Proceedings of the Annual Meeting of the German Association for Pattern Recognition (DAGM '05), Wien, Austria, Aug/Sept. 2005 [PDF]
Simone Frintrop, Erich Rome, Andreas Nüchter, and Hartmut Surmann: A Bimodal Laser-Based Attention System, Journal of Computer Vision and Image Understanding (CVIU), vol. 100, no. 1-2, Oct-Nov 2005, Special Issue on Attention and Performance [PDF]