Prof. Dr. Dr.-Ing. Wolfgang Minker
Universität Ulm
Abteilung Informationstechnik

New Research Directions for Multimodal Dialogue Systems - User-Friendliness, Intelligence, and Adaptiveness

User-friendly Multimodal Dialogue Systems (MDSs), with which the user can have access to computer-based applications using different interaction modalities such as speech, gestures, etc., have been an active research area for many years.

However, many existing MDSs are either very limited in the scope of domain functionality or require a rather cumbersome interaction. With more and more application domains becoming available, ranging from unified messaging to trip planning and appointment scheduling, it is obvious that the current interfaces need to be rendered more efficient. In particular, the possibility to construct and to manage complex tasks and interdependencies with these applications requires a high cognitive burden from the user, which may be dangerous in certain environments (e.g., driver distraction). However, rather than prevent the use of such applications, it seems preferable to relieve the user as much as possible from the need to manage the complexities involved alone in his mind.

Empirical studies also show that users tend to apply social norms to computers. They blame them for mistakes, try to be polite if the computer asks them for their opinion and feel flattered if the computer compliments them on a good performance. In order to build MDSs that are able to communicate with the user in a more natural way, the consideration of social aspects is inevitable.

In my talk, I will revisit some of our past and ongoing projects and research initiatives in the domain of user-friendly, intelligent and adaptive MDSs.