Accession Number:

ADA263301

Title:

An Evaluation Methodology for Natural Language Processing Systems

Descriptive Note:

Final rept. Jan 1991-Jul 1992

Corporate Author:

CALSPAN CORP BUFFALO NY

Report Date:

1992-12-01

Pagination or Media Count:

338.0

Abstract:

The Neal-Montgomery NLP Evaluation Methodology was developed under the Benchmark InvestigationIdentification project as a means of determining the linguistic competence of Natural Language Processing NLP systems. Embodied in an evaluation tool based on a detailed classification of linguistic phenomena with, currently, over 350 test items, the methodology produces descriptive profiles of NLP system linguistic capabilities, and can be applied without regard to the current system application, domain, or specific language processing function. To date, it has been applied to three NL data base query systems and three text processing systems. Researchers are invited to apply the evaluation methodology included within this report to their systems and report their critique and recommendations towards the evolution of standard evaluation procedures for NLP systems.

Subject Categories:

  • Linguistics
  • Computer Programming and Software

Distribution Statement:

APPROVED FOR PUBLIC RELEASE