AAAI Publications, The Twenty-Sixth International FLAIRS Conference

Font Size: 
Acquiring User Models to Test Automated Assistants
Marc Pickett, David W. Aha, J. Gregory Trafton

Last modified: 2013-05-19

Abstract


A central problem in decision support tasks is operator overload, in which a human operator's performance suffers because he or she is overwhelmed by the cognitive requirements of a task. To alleviate this problem, it would be useful to provide the human operator with an automated assistant to share some of the task's cognitive load. However, the development cycle for building an automated assistant is hampered by the testing phase because this involves human user studies, which are costly and time-consuming to conduct. As an alternative to user studies, we propose acquiring user models, which can be used as a proxy for human users during middle iterations, thereby significantly shortening the development cycle for rapid development. The primary contribution of this paper is a method for coarsely testing automated assistants by using user models acquired from traces gathered from various individual human operators. We apply this method in a case study in which we evaluate an automated assistant for users operating in a simulation of multiple unmanned aerial vehicles.

Full Text: PDF