Einladung zur hochschulöffentlichen Disputation

von Herrn Christopher Baumgärtner

Dienstag, 2. Juli 2013, 16:15 Uhr

Informatikum, Vogt-Kölln-Str. 30, Haus F, Raum 534

“On-Line Cross-Modal Context Integration for Natural Language Parsing”

Abstract:

Natural language processing can benefit from the integration of information received from other channels. We present a computational model for merging information from the visual and the natural language modalities. Depending on the perceived visual context processing of sentences is changed accordingly.

The context that is perceived through the visual channel oftentimes contains entities that do not apply to a given sentence. Integrating these would result in incorrect processing results of the system. The system therefore limits processing of context to those parts that seem to be relevant for processing a given sentence. This choice of a subset of context information is dependent on the information received from the language modality. As this changes due to the integration of context, the interaction between the subsystems of language and vision becomes bi-directional as language processing in-fluences the choice of visual information to be integrated, which in turn influences language processing.

Our system realizes this behavior by modeling attentional processes found in the human cognitive system. Visual attention can be divided into bottom-up and top-down processes. Bottom-up attention depends exclusively on features of sensory input while top-down attention is imposed by cognitive processes not directly linked to visual features. Language input can be a driving factor of top-down attention by influencing listeners to show a goal directed behavior after hearing words and phrases referring to objects in the visual field. Modeling these processes is helpful when choosing the above-mentioned subset of contextual entities that applies to a given sentence. We evaluated the system by comparing its behavior with results from experiments investigating the integration of langu-age and vision in humans.

Kontakt:

Prof. Dr. Christopher Habel
Vorsitzender des Fach-Promotionssausschusses Informatik