Generating spatial descriptions for cross-modal references

Abstract

We present a localisation component that supports the generation of cross-modal deictic expressions in the knowledge-based presentation system WIP. We deal with relative localisations (e.g., "The object to the left of object X.';), absolute localisations (e.g., "The object in the upper left part of the picture.';) and corner localisations (e.g., "The object in the lower right corner of the picture';). In addition, we distinguish two localisation granularities, one less detailed (e.g., "the object to the left of object X.';) and one more detailed (e.g., "the object above and to the left of object X.';). We consider corner localisations to be similar to absolute localisations and in turn absolute localisations to be specialisations of relative localisations. This allows us to compute all three localisation types with one generic localisation procedure. As elementary localisations are derived from previously computed composite localisations, we can cope with both localisation granularities in a computationally efficient way. Based on these primary localisation procedures, we discuss how objects can be localised among several other objects. Finally we introduce group localisations (e.g., "The object to left of the group of other objects.';) and show how to deal with them.