Research Projects

TRAIL is a new Doctoral Network project as Marie Skłodowska-Curie Action funded by the European Commission 1.3.2023-28.2.2027.

Abstract

TRAIL strategically focuses on a novel, highly interdisciplinary and cross-sectorial research and training programme for a better understanding of transparency in deep learning, artificial intelligence and robotics systems. In order to train a new generation of Early Stage Researchers (ESR) to become experts in the design and implementation of transparent, interpretable neural systems and robots, we have built a highly interdisciplinary consortium, containing expert partners with long-standing expertise in cutting-edge artificial intelligence and robotics, including deep neural networks, computer science, mathematics, social robotics, human-robot interaction and psychology. In order to build transparent robotic systems, these new ESR researchers need to learn about the theory and practice of the principles of (1) internal decision understanding and (2) external transparent behaviour. Since the ability to interpret complex robotic systems needs highly interdisciplinary knowledge, we will start, on the decision level, to interpret deep neural learning and analyse what knowledge can be efficiently extracted. At the same time, on the behaviour level, the disciplines of human-robot interaction and psychology will be key in order to understand how to present the extracted knowledge as behaviour in an intuitive and natural way to a human user to integrate the robot into a cooperative human-robot interaction.

A scaffolded training curriculum will guarantee that the ESRs have not only a deep understanding of both research areas, but experience optimal skill training to be fully prepared for a successful research career in academia and industry. The importance and need of this research for the industry is clearly visible with the full commitment of 7 leading European and world-wide-operating robotics companies that together cover the majority of Europe’s robot market and a broad spectrum of AI applications.

PI: Prof. Dr. Stefan Wermter
Associate: Theresa Pekarek Rosin, Sergio Lanza, Julia Gachot

Details:
TRAIL project

Partners:

Cognitive Systems, Comenius University in Bratislava Slovakia
Knowledge Technology, Universität Hamburg, Germany
IIT Istituto Italiano di Tecnologia, Italy

The TERAIS project aims at collaborating with the Department of Applied Informatics at Comenius University in Bratislava to become a workplace of international academic excellence in Robotics and Artificial Intelligence. To achieve this, the project aims to (1) prepare conditions for systematic development of research capacity in Artificial Intelligence and Robotics, (2) establish sustainable networking and collaboration links with partner institutions Comenius University in Bratislava, Universität Hamburg and the Italian Institute of Technology, in order to boost institutional international research at Comenius University.

The partnership will comprise, in addition to the Department of Applied Informatics at Comenius University in Bratislava, the Knowledge Technology team at Universität Hamburg - with excellent expertise in neural networks, deep learning and crossmodal learning for developmental robotics - and two units at Italian Institute of Technology in Genoa - with excellent experience in humanoid robotics, control using physical and simulated robots and human-robot interaction. The project is expected to contribute to promoting research excellence in AI and Robotics in Comenius University in Bratislava based on the expertise from the Knowledge Technology Team in Hamburg and the IIT Team in Genoa.

Researchwise, the TERAIS project spans the areas of advancement of cognitive robotics and human–robot interaction which are considered aiming to improve the quality of life. Constructivist computational approaches (understanding cognition by building artifacts) are expected to play an increasingly more important role in the future. Artificial neural networks, as a successful machine learning approach, are primarily used as building AI blocks for robots. Cognitive robotics stands on three pillars (research areas) - artificial intelligence (AI), cognitive science and robotics. In this process, cognitive science will very likely pave its way to help build more explainable, trustworthy and sense-making AI solutions. Flexibility and versatility is even more articulated in cognitive robotics, enabling truly multimodal flexible robots deployable for human-oriented applications and human-robot interaction.

PI: Prof. Dr. Stefan Wermter
Associate: Hassan Ali, Dr. Matthias Kerzel, Dipl.-Ing. Erik Strahl, Dr. Cornelius Weber

Details:
TERAIS project
http://www.terais.eu/

SIDIMO is an R&D cooperation project within the ZIM-network “DEMOBIS”. In collaboration with our project partner CIBEK, this project aims to develop a robust German speech recognition system for elderly people in a smart-home environment.

Systems that support elderly people in their home environments offer their target group an extension on an independent and self-determined way of living. However, existing speech recognition systems for home environments usually transfer recorded data to third-party servers, which raises a series of privacy issues. Additionally, while language as an intuitive means of communication can facilitate interactions of inexperienced users with assistance systems, ASR systems often struggle with elderly voices due to irregularities in speech patterns or indistinct pronunciations.

The goal of SIDIMO is to develop a speech recognition system, which 1) adjusts existing network architectures for German ASR to work locally, keeping private data in the possession of the user, and 2) can continually adapt to the changing speech of elderly users to increase speech recognition accuracy.

Duration: 01.10.2021 - 30.09.2023
PI: Prof. Dr. Stefan Wermter
Associate: Theresa Pekarek Rosin, Dr. Matthias Kerzel

Details: SIDIMO project

With the tremendous success of data-driven approaches in pattern recognition, natural language processing and decision-making for robotic tasks, the debate on the poor interpretability of the “black-box” model becomes more intense. The problem of non-interpretability impacts their applicability and has therefore become a major research focus. Taking image classification on ImageNet as an example, although a simple ResNet-based classifier can readily achieve a very high accuracy that was unattainable just a few years ago, the complexity within its highly tangled network modules makes it impossible to prove the robustness and performance of its classification given unseen data. Hence, a huge demand arises for tools to structurally inspect the learned “knowledge” of a neural network. Emergent post-hoc visualization techniques, like CAM, LIME and Grad-CAM attempt to significantly improve explainability, by visually making predictions from Convolutional Neural Networks (CNNs) more transparent, though the full CNN-based model itself is usually uninterpretable. The challenge of the VeriKAS project is that the computations of the model can be better interpreted by humans in an appropriate manner. The VeriKAS project is a BMWi funded project that aims to realize robust explanation and future certification of deep learning-based systems, in collaboration with our project partners ZAL, HITeC and hs2.

Pl: Prof. Dr. Stefan Wermter
Associates: Dr. Sven Magg, Dr. Matthias Kerzel, Wenhao Lu

Details: VeriKAS project

What are the principal mechanisms required to capture the robustness and interactivity of human communication, given the situational, noisy and often ambiguous nature of natural language? And how, and to what extent, can we integrate these mechanisms within an embodied functional model that is computationally and empirically verifiable on a physical robot?

We address these research questions by investigating the linguistic phenomenon of conversational repair (CR) – a method to edit and to re-interpret previously uttered sentences that were not correctly understood by the hearer. As an example, consider the following scenario: 

A human operator issues the underspecified command “Pick up the apple”. The operator intends to let the robot pick up the red apple, but the robot assumes that the human refers to the green apple (possibly because it is closer to the robot). As it picks up the green apple, the observed course of actions deviates from the user’s intended course of actions. Shortly after the human recognizes this error, he utters a repair command that lets the robot pick up the red apple.

Our own and other existing computational models for human-robot dialog consider conversational repair in the case of non-understandings, but they do not consider misunderstandings. In the LeCAREbot project, we will investigate how misunderstandings can be addressed in human-robot dialog. Herein, we develop compositional state representations for mixed verbal-physical interaction states, and we will develop a hybrid neuro-symbolic approach to learn such state abstractions. We will combine the state abstraction with model-based hierarchical reinforcement learning to realize robotic scenarios where conversational repair plays a critical role. 

PIs: Dr. Manfred Eppe, Prof. Dr. Stefan Wermter

Details: LeCAREBot project

Humans possess very sophisticated learning mechanisms that allow them, for example, to learn a sports discipline or task, and to transfer certain movement patterns and skills from this discipline to improve their performance in another discipline or task. This is possible, even if the transfer task is not immediately related to the initially learned task. It is this important transfer learning ability that enables humans to solve problems they have never encountered before, in ways that are beyond the current capabilities of robots and artificial agents. However, the neural foundations of transfer learning and its role in the emergence of a self are still mostly unknown, and there exists no generally accepted functional neural model of transfer learning.

Dr. Manfred Eppe and Prof. Stefan Wermter from the Knowledge Technology group have been awarded a grant within the DFG priority programme The Active Self to address this issue. 

PIs: Dr. Manfred Eppe, Prof. Dr. Stefan Wermter

Details: IDEAS project

What are the neuro-computational principles of the advanced problem-solving capabilities that we observe in higher animals like corvids and primates, and how can we use those principles as the basis for a computational model in an acting robot? 

Dr. Manfred Eppe and Prof. Stefan Wermter from the Knowledge Technology group have been awarded a grant within the Experiment! programme of the Volkswagen Stiftung (acceptance rate for this programme is below 5%) to address this question within the project "The Neural Microcircuits of Problem-solving and Goal-directed Behavior".

The aim of this high-risk/high-gain project is to design and build a computational model of neural problem-solving circuits, and to evaluate it with an embodied problem-solving robot. The results will help to understand goal-directed behavior of animals, and they are expected to create a novel breakthrough for the development of intelligent goal-directed causal robotic agents. The model will be realized and evaluated on a physical robotic agent that shall be able to solve problems like the rocker-problem, which involves abstract reasoning about concepts like friction, force, and gravity. 

PIs: Dr. Manfred Eppe, Prof. Dr. Stefan Wermter

Details: Neural problem-solving project