Towards a plan-based synthesis of illustrated documents

Abstract

A major drawback of existing systems for the synthesis of multimodal documents is that they generate textual and graphical parts mostly independently of each other. Consequently, the relation between text and graphics remains opaque in such documents. This report starts from the assumption that not only the generation of text, but also the generation of multimodal documents can be considered as a communicative act in the achievement of certain goals. A plan-based approach seems adequate for the realization of a system able to automatically generate illustrated documents. First, we show that the distinction between main and subsidiary acts proposed by textlinguists is also suitable for text-picture-combinations. Starting from this distinction, we formulate strategies which relate both to text and picture production. The joint planning of text and pictures is regarded as a fundamental prerequisite for the coordination of different modes.