An Evaluation Methodology for Natural Language Processing Systems
Final rept. Jan 1991-Jul 1992
CALSPAN CORP BUFFALO NY
Pagination or Media Count:
The Neal-Montgomery NLP Evaluation Methodology was developed under the Benchmark InvestigationIdentification project as a means of determining the linguistic competence of Natural Language Processing NLP systems. Embodied in an evaluation tool based on a detailed classification of linguistic phenomena with, currently, over 350 test items, the methodology produces descriptive profiles of NLP system linguistic capabilities, and can be applied without regard to the current system application, domain, or specific language processing function. To date, it has been applied to three NL data base query systems and three text processing systems. Researchers are invited to apply the evaluation methodology included within this report to their systems and report their critique and recommendations towards the evolution of standard evaluation procedures for NLP systems.
- Computer Programming and Software