Expressing Emotions: Using Symbolic and Parametric Gestures in Interactive Systems

Paul Modler

In this paper we present a system that maps hand gestures into musical parameters in an interactive computer music performance and virtual reality environment. In the first part of the paper we comment on our view of emotions. Thereafter, the technical back-ground will be introduced We show that in a performing situation the expres-sion of emotion is strongly related to intuitive and interactive aesthetic variations. Our focus on the mapping of these gestural varia-tions into relevant musical parameters leads to the concept of symbolic and parametric subgestures. This allows the generation of emotional and aes-thetic variations. We use data obtained from a sensor glove, a high end input device for digitizing hand and finger motions into multi-parametric data. The processing of this data is done by a neural network recognizing symbolic subgestures com-bined with the extraction of parametric values. We present a dictionary of symbolic and paramet-ric subgestures which is categorised with respect to complexity. The system is complemented with a 3D VRML environment, i.e. an animated hand model and be-having representations of musical structures. This 3D representation combines with the gesture proc-essing module and the sound generation to so called "Behaving Virtual Musical Objects."


This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.