Towards an understanding of coherence in multimodal discourse

Abstract

An understanding of coherence is attempted in a multimodal framework where the presentation of information is composed of both text and picture segments (or, audio-visuals in general). Coherence is characterised at three levels: coherence at the syntactic level which concerns the linking mechanism of the adjacent discourse segments at the surface level in order to make the presentation valid; coherence at the semantic level which concerns the linking of discourse segments through some semantic ties in order to generate a wellformed thematic organisation; and, coherence at the pragmatic level which concerns effective presentation through the linking of the discourse with the addressees' preexisting conceptual framework by making it compatible with the addressees' interpretive ability, and linking the discourse with the purpose and situation by selecting a proper discourse typology. A set of generalised coherence relations are defined and explained in the context of picture-sequence and multimodal presentation of information.