Informatisches Kolloquium &
Kolloquium des Internationalen Graduiertenkollegs CINACS
“Cross-Modal Interactions in Natural and Artificial Cognitive Systems”

Montag, 30. April 2007

Prof. Matthew Crocker
Universität Saarbrücken

The Coordinated Interplay of Visual Attention and Utterance Comprehension

There is substantial evidence that language comprehension mechanisms interact closely with visual attention processes during situated language processing. In this talk, I will present findings from a collection of recent experiments using the “visual world” paradigm, which monitors eye movements in scenes during spoken comprehension. I will argue not only that incremental utterance interpretation drives attention to relevant scene areas, but also that the attended scene information can in turn immediately influence our understanding of what we have heard. Our recent findings also indicate that eye gaze can be directed towards the prior location of objects and events which are no longer present, supporting the notion of “spatial indexing”, and the importance of eye-movements in re-accessing information even when it is no longer visible. These findings have led to the proposal of the “coordinated interplay account” (CIA) of scene and utterance processing, which aims to explicitly outline the mechanisms involved, and explain utterance-mediated eye-movement behavior. Time permitting, I will then briefly outline a connectionist model that has been developed to instantiate the CIA, including an explicit visual attention mechanism.

Kontakt

Prof. Dr. Wolfgang Menzel
Telefon: 2435