Universität Hamburg - der Forschung, der Lehre, der Bildung, zur Homepage
Computer Vision (CV)

Photo: UHH/Lutsch

Computer Vision

Multi-modal/Audio-Visual Signal Processing

Our world is complex, and using different senses helps humans to grasp different aspects of the world. Similarly, investigating multiple data modalities enables AI systems to exploit the advantages of each modality. Since modalities are often complementary, this offers plenty of opportunities to improve the system performance. We investigate and combine different modalities, such as audio-visual data or depth data to face the challenges of real-world data.

Our corresponding publications can be found below.

Publications

  • Danu Caus, Guillaume Carbajal, Timo Gerkmann, Simone Frintrop: See the silence: improving visual-only voice activity detection by optical flow and RGB fusion, International Conference on Computer Vision Systems (ICVS) 2021, [PDF]
  • Quan Nguyen, Julius Richter, Mikko Lauri, Timo Gerkmann, Simone Frintrop: Improving mix-and-separate training in audio-visual sound source separation with an object prior, ICPR 2020, [PDF]
  • Dominik Klein, Dirk Schulz, and Simone Frintrop: Boosting with a Joint Feature Pool from Different Sensors, 7th International Conference on Computer Vision Systems (ICVS 2009), Oct. 2009 [PDF]
  • Simone Frintrop, Erich Rome, Andreas Nüchter, and Hartmut Surmann: A Bimodal Laser-Based Attention System, Journal of Computer Vision and Image Understanding (CVIU), vol. 100, no. 1-2, Oct-Nov 2005, Special Issue on Attention and Performance. [PDF]
  • S. Frintrop; E. Rome; A. Nüchter; H. Surmann Applying Attentional Mechanisms to Bi-modal 3D Laser Data. In: International Workshop on Attention and Performance in Computer Vision (WAPCV 2003), L. Paletta, G.W. Humphreys, and R.B. Fisher (eds), pp. 25-30, Conference: Graz, Austria, April 3, 2003 [PDF]
  • S. Frintrop; E. Rome; A. Nüchter; H. Surmann An Attentive, Multi-modal Laser "Eye". In: Proc. of 3rd International Conference on Computer Vision Systems (ICVS 2003), J. Crowley, J.H. Piater, M. Vincze, and L. Paletta (eds), pp. 202-211, Springer, Berlin, LNCS 2626, 2003. Conference: Graz, Austria, April 1-2, 2003 Abstract | [PDF] © Springer Verlag