Measuring Human Readability of Machine Generated Text: Three Case Studies in Speech Recognition and Machine Translation
MASSACHUSETTS INST OF TECH LEXINGTON LINCOLN LAB
Pagination or Media Count:
We present highlights from three experiments that test the readability of current state-of-the art system output from 1 an automated English speech-to-text system 2 a textbased Arabic-to-English machine translation system and 3 an audiobased Arabic-to-English MT process. We measure readability in terms of reaction time and passage comprehension in each case, applying standard psycholinguistic testing procedures and a modified version of the standard Defense Language Proficiency Test for Arabic called the DLPT. We learned that 1 subjects are slowed down about 25 when reading system STT output, 2 textbased MT systems enable an English speaker to pass Arabic Level 2 on the DLPT and 3 audiobased MT systems do not enable English speakers to pass Arabic Level 2. We intend for these generic measures of readability to predict performance of more application specific tasks.
- Voice Communications