Social Interaction: Multimodal Conversation with Social Agents

Katashi Nagao, Akikazu Takeuchi

We present a new approach to human-computer interaction, called social interaction. Its main characteristics are summarized by the following three points. First, interactions are realized as multimodal (verbal and nonverbal) conversation using spoken language, facial expressions, and so on. Second, the conversants are a group of humans and social agents that are autonomous and social. Autonomy is an important property that allows agents to decide how to act in an ever-changing environment. Socialness is also an important property that allows agents to behave both cooperatively and collaboratively. Generally, conversation is a joint work and ill-structured. Its participants are required to be social as well as autonomous. Third, conversants often encounter communication mismatches (misunderstanding others’ intentions and beliefs) and fail to achieve their joint goals. The social agents, therefore, are always concerned with detecting communication mismatches. We realize a social agent that hears human-to-human conversation and informs what is causing the misunderstanding. It can also interact with humans by voice with facial displays and head (and eye) movement.


This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.