Please use this identifier to cite or link to this item: doi:10.22028/D291-28451
Title: The influence of visual information on word predictability and processing effort
Author(s): Ankener, Christine Susanne
Language: English
Year of Publication: 2019
Free key words: index of cognitive activity
eye tracking
N 400
situated language processing
DDC notations: 400 Language, linguistics
Publikation type: Dissertation
Abstract: A word’s predictability or surprisal in linguistic context, as determined by cloze probabilities or languagemodels (e.g.,Frank,2013a) is related to processing effort, in that less expected words take more effort to process (e.g., Hale, 2001). This shows how, in purely linguistic contexts, rational approaches have been proven valid to predict and formalise results from language processing studies. However, the surprisal (or predictability) of a word may also be influenced by extra-linguistic factors, such as visual context information, as given in situated language processing. While, in the case of linguistic contexts, it is known that the incrementally processed information affects the mental model (e.g.,Zwaan and Radvansky, 1998) at each word in a probabilistic way, no such observations have been made so far in the case of visual context information. Although it has been shown that in the visual world paradigm (VWP), anticipatory eye movements suggest that listeners exploit the scene to predict what will be mentioned next (Altmann and Kamide,1999), it is so far unclear how visual information actually affects expectations for and processing effort of target words. If visual context effects on word processing effort can be observed, we hypothesise that rational concepts can be extended in order to formalise these effects, hereby making them statistically accessible for language models. In a line of experiments, I hence observe how visual information – which is inherently different from linguistic context, for instance in its non-incremental-at once-accessibility– affects target words. Our findings are a clear and robust demonstration that the non-linguistic context can immediately influence both lexical expectations, and surprisal-based processing effort as assessed by two different on-line measures of effort (a pupillary and an EEG one). Finally, I use surprisal to formalise the measured results and propose an extended formula to take visual information into account.
Link to this record: urn:nbn:de:bsz:291--ds-284518
Advisor: Staudte, Maria
Date of oral examination: 24-Jun-2019
Date of registration: 26-Sep-2019
Third-party funds sponsorship: Deutsche Forschungsgemeinschaft (DFG), Grant SFB 1102
Sponsorship ID: 1102
Faculty: P - Philosophische Fakultät
Department: P - Sprachwissenschaft und Sprachtechnologie
Collections:SciDok - Der Wissenschaftsserver der Universität des Saarlandes

Files for this record:
File Description SizeFormat 
thesis_VÖ.pdfVolltext Dissertation "The Influence of Visual Information on Word Predictability and Processing Effort", inkl. Anhang38,69 MBAdobe PDFView/Open

Items in SciDok are protected by copyright, with all rights reserved, unless otherwise indicated.