Representations for Multi-Modal Human-Computer Interaction
Papers from the AAAI Workshop
Syed Ali and Susan McRoy, Program Cochairs
Technical Report WS-98-09 published by The AAAI Press, Menlo Park, California
This technical report is also available in book and CD format.
Please Note: Abstracts are linked to individual titles, and will appear in a separate browser window. Full-text versions of the papers are linked to the abstract text. Access to full text may be restricted to AAAI members. PDF file sizes may be large!
Contents
Efficient Representations for Multi-Modal Interaction / 1
Syed S. Ali and Susan W. McRoy
Generating Coordinated Natural Language and 3D Animations for Complex Spatial Explanations / 5
Charles B. Callaway, Stuart G. Towns, and James C. Lester
Building Multimodal Systems: Compromise between Theory and Practice / 11
Marilyn Cross, Christian Matthiessen, Licheng Zeng, and Ichiro Kobayashi
Generating Visual Arguments: a Media-Independent Approach / 17
Nancy Green, Stephan Kerpedjiev, Steven F. Roth, Giuseppe Carenini, and Johanna Moore
The Spoken Language Navigation Task / 21
J. Gurney, E. Kipple, and T. Gregory
A Human-Computer Interaction Framework for Media-Independent Knowledge / 26
Robert J. K. Jacob and James G. Schmolze
Multimodal Unification-based Grammars / 31
Michael Johnston
On Representing Salience and Reference in Multimodal Human-Computer Interaction / 33
Andrew Kehler, Jean-Claude Martin, Adam Cheyer, Luc Julia, Jerry R. Hobbs, and John Bear
Mixed Depth Representations in the B2 System / 40
Susan W. McRoy, Susan M. Haller, and Syed S. Ali
Multi-Modal Seamless Messaging / 46
John F. Meech and Suhayya Abu-Hakima
A Representation of Media for Multimedia Authoring and Browsing Systems / 49
Ethan V. Munson and Mark Pfeiffer
Multimodal Prediction and Classification of Audio-Visual Features / 55
Vladimir Pavlovic and Thomas S. Huang
Interactional Competency for Conversational Characters / 61
Scott Prevost, Timothy Bickmore, and Justine Cassell
Task-Oriented Dialogs with Animated Agents in Virtual Reality / 64
Jeff Rickel and W. Lewis Johnson
Mapping Between Image Regions and Caption Concepts of Captioned
Depictive Photographs / 70
Neil C. Rowe
Integrating Different Knowledge Representations in an Intelligent System:
Standardization Allows Diversity? / 76
Ronnie Smith
Cognitive Constraints on the Use of Visible Speech and Gestures / 78
Laura A. Thompson
A Statistical Approach to Multimodal Natural Language Interaction / 81
John Vergo
Towards an Ontology for Library Modalities / 86
Christopher A. Welty
Acting on a Visual World: the Role of Perception in Multimodal HCI / 89
Frederic Wolff, Antonella De Angeli, and Laurent Romary
AAAI Digital Library
AAAI relies on your generous support through membership and donations. If you find these resources useful, we would be grateful for your support.