CHROME Corpus: Multi-level annotation of an audiovisual corpus

In this paper we describe the methodology employed for the annotation of a resource developed within the CHROME (Cultural Heritage Resources Orienting Multimodal Experience) project, aimed at the protection and promotion of cultural heritage. More specifically, the ultimate goal of the project is the modelling of multimodal data (including speech features and gestures) for the design of a virtual agent serving in museums and capable of communicating in intelligible as well as effective and natural way. In order to grasp the relationship between linguistic and gestural behaviours, multi-level annotation systems have been developed and implemented for the labelling of linguistic and gesture features on different levels of analysis. This article is dedicated to a general presentation of the corpus and to the description of the different levels of linguistic annotation; then, the final section, reports conclusive remarks considering the applications of the described methodology. The CHROME corpus and the mark-up methodology described in this work represent valuable multimodal resources for investigations on communicative dynamics which may offer valid support for both theoretical and practical applications.