University of Hamburg - to research, to teach, to educate and form, to homepage
Knowledge Technology
Department of InformaticsKnowledge Technology

Photo: UHH/Knowledge Technology

Research

LingoRob - Learning Language in Developmental Robots

LingoRob is a DAAD funded project between our project partners at Inria, Bordeaux and the Knowledge Technology group. The scientific objective of the collaboration is to better understand the mechanisms underlying language acquisition and enable more natural interaction between humans and robots in different languages, while modeling how the brain processes sentences and integrates semantic information of scenes. Models developed in both labs involve artificial neural networks, and in particular Echo State Networks (ESN), also known as pertaining to the Reservoir Computing framework. These neural models allow insights on high-level processes of the human brain, and at the same time are well suited as robot control platform, because they can be trained and executed online with low computational resources. The object localization and recognition is implemented using computer vision approaches as a baseline and using deep learning methods to progress on the generalisation of the scenario.

Programme: DAAD, Programm Projektbezogener Personenaustausch Frankreich 2017
Duration 01.01.2017 - 31.12.2018

PIs: Dr. Xavier Hinaut, Dr. Cornelius Weber, Prof. Dr. Stefan Wermter
Staff: Johannes Twiefel, Dr. Doreen Jirak

Published Outcomes

  • Johannes Twiefel, Xavier Hinaut, and Stefan Wermter. “Syntactic Reanalysis in Language Models for Speech Recognition”. In: 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob). 2017.
  • Nikhil Churamani, Paul Anton, Marc Brügger, Erik Fliesswasser, Thomas Hummel, Julius Mayer, Waleed Mustafa, Hwei Geok Ng, Thi Linh Chi Nguyen, Quan Nguyen, Marcus Soll, Sebastian Springenberg, Sascha Griffiths, Stefan Heinrich, Nicolás Navarro-Guerrero, Erik Strahl, Johannes Twiefel, Cornelius Weber, and Stefan Wermter. “The Impact of Personalisation on Human-Robot Interaction in Learning Scenarios”. In: Proceedings of the Fifth International Conference on Human Agent Interaction. Bielefeld, Germany: ACM, 2017, accepted.
  • Hwei Geok Ng, Paul Anton, Marc Brügger, Nikhil Churamani, Erik Fliesswasser, Thomas Hummel, Julius Mayer, Waleed Mustafa, Thi Linh Chi Nguyen, Quan Nguyen, Marcus Soll, Sebastian Springenberg, Sascha Griffiths, Stefan Heinrich, Nicolás Navarro-Guerrero, Erik Strahl, Johannes Twiefel, Cornelius Weber, and Stefan Wermter. “Hey Robot, Why Don’t You Talk To Me?” In: Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). Aug. 2017, pp. 728–731.
  • Marian Tietz, Tayfun Alpay, Johannes Twiefel, and Stefan Wermter. “Semi-Supervised Phoneme Recognition with Recurrent Ladder Networks”. In: International Conference on Artificial Neural Networks. Springer. 2017, pp. 3–10.
  • Marian Tietz. “Semi-Supervised Learning with Recurrent Ladder Networks”. In: Universität Hamburg, Dept. Informatik, Master’s Thesis (2016).
  • Surender Kumar. “Word2Vec and Echo State Network For Thematic Role Assignment”. In: Universität Hamburg, Dept. Informatik, Master’s Thesis (2016).
  • Kamila Ignatowicz. “Improving the Quality of Crowdsourced Data Acquisition”. In: Universität Hamburg, Dept. Informatik, Bachelor’s Thesis (2017).
  • Julian Tobergte. “Improved Interpretation of Robot Commands via Crossmodal Validation with a Spatial Planner”. In: Universität Hamburg, Dept. Informatik, Bachelor’s Thesis (2017).
  • Xavier Hinaut and Johannes Twiefel. “Teach Your Robot Your Language! Trainable Neural Parser for Modelling Human Sentence Processing: Examples for 15 Languages”. In: IEEE Transactions on Cognitive and Developmental Systems (2019), to appear.