Accession Number : ADA263301


Title :   An Evaluation Methodology for Natural Language Processing Systems


Descriptive Note : Final rept. Jan 1991-Jul 1992


Corporate Author : CALSPAN CORP BUFFALO NY


Personal Author(s) : Neal, Jeannette G ; Feit, Elissa L ; Funke, Douglas J ; Montgomery, Christine A


Full Text : https://apps.dtic.mil/dtic/tr/fulltext/u2/a263301.pdf


Report Date : Dec 1992


Pagination or Media Count : 338


Abstract : The Neal-Montgomery NLP Evaluation Methodology was developed under the 'Benchmark Investigation/Identification' project as a means of determining the linguistic competence of Natural Language Processing (NLP) systems. Embodied in an evaluation tool based on a detailed classification of linguistic phenomena with, currently, over 350 test items, the methodology produces descriptive profiles of NLP system linguistic capabilities, and can be applied without regard to the current system application, domain, or specific language processing function. To date, it has been applied to three NL data base query systems and three text processing systems. Researchers are invited to apply the evaluation methodology (included within this report) to their systems and report their critique and recommendations towards the evolution of standard evaluation procedures for NLP systems.


Descriptors :   *COMPUTATIONAL LINGUISTICS , *NATURAL LANGUAGE , TEST AND EVALUATION , DATA BASES , METHODOLOGY , TEXT PROCESSING , COMPUTER BENCHMARKING , TOOLS , PROFILES , CLASSIFICATION , STANDARDS


Subject Categories : Linguistics
      Computer Programming and Software


Distribution Statement : APPROVED FOR PUBLIC RELEASE