Adelheit Stein, Ulrich Thiel
We propose a comprehensive framework for modeling and specifying multimodal interactions. To this end, we employ an extended notion of ' dialogue acts’ which can be realized by linguistic and non-linguistic means. First, a set of constraints is presented that describes the temporal structure and all patterns of exchange during a cooperative information-seeking dialogue. Second, we introduce a strategic level of description which allows the specification of the topical structure according to an information-seeking strategy. The model was used to design and implement the MERIT system, and led to a reduction in the complexity of the user interface while preserving most of the useful, but sometimes confusing, dialogue options of advanced direct manipulation interfaces.