Accession Number:

ADA533887

Title:

Assessing the Speaker Recognition Performance of Naive Listeners Using Mechanical Turk

Descriptive Note:

Technical paper Oct-Nov 2010

Corporate Author:

MASSACHUSETTS INST OF TECH LEXINGTON LINCOLN LAB

Report Date:

2010-10-25

Pagination or Media Count:

5.0

Abstract:

In this paper we attempt to quantify the ability of naive listeners to perform speaker recognition in the context of the NIST evaluation task. We describe our protocol a series of listening experiments using large numbers of naive listeners 432 on Amazons Mechanical Turk that attempt to measure the ability of the average human listener to performance speaker recognition. Our goal was the compare the performance of the average human listener to both forensic experts and state-of-the-art automatic systems. We show that naive listeners vary substantially in their performance, but that a voting of listeners can achieve performance similar to that of expert forensic examiners.

Subject Categories:

  • Voice Communications

Distribution Statement:

APPROVED FOR PUBLIC RELEASE