Acquiring User Models to Test Automated Assistants
NAVAL RESEARCH LAB WASHINGTON DC NAVY CENTER FOR APPLIED RESEARCH IN ARTIFICIAL INTELLIGENCE
Pagination or Media Count:
A central problem in decision support tasks is operator overload in which a human operators performance suffers because he or she is overwhelmed by the cognitive requirements of a task. To alleviate this problem, it would be useful to provide the human operator with an automated assistant to share some of the tasks cognitive load. However, the development cycle for building an automated assistant is hampered by the testing phase because this involves human user studies which are costly and time-consuming to conduct. As an alternative to user studies, we propose acquiring user models which can be used as a proxy for human users during middle iterations, thereby significantly shortening the development cycle for rapid development. The primary contribution of this paper is a method for coarsely testing automated assistants by using user models acquired from traces gathered from various individual human operators. We apply this method in a case study in which we evaluate an automated assistant for users operating in a simulation of multiple unmanned aerial vehicles.