Accession Number:

AD1158500

Title:

Evaluating Subjective Feedback for Internet of Things Dialogues

Descriptive Note:

[Technical Report, Research Paper]

Corporate Author:

UNIVERSITY OF SOUTHERN CALIFORNIA LOS ANGELES

Report Date:

2018-01-01

Pagination or Media Count:

9

Abstract:

This paper discusses the process of determining which subjective features are seen as ideal in a dialogue system, and linking these features to objectively quantifiable behaviors. A corpus of simulated system-user dialogues in the Internet of Things domain was manually annotated with a set of system communicative and action responses, and crowd-sourced ratings and qualitative feedback of these dialogues were collected. This corpus of subjective feedback was analyzed, revealing that raters described top ranked dialogues as Intelligent, Natural, Pleasant, and as having Personality. Additionally, certain communicative and action responses were statistically more likely to be present in dialogues described as having these features. There was also found to be a lack of agreement among raters as to whether a direct communication style, or a conversation alone was preferred, suggesting that future research and development should consider creating models for different communication styles.

Subject Categories:

  • Psychology
  • Linguistics

Distribution Statement:

[A, Approved For Public Release]