Accession Number:



BBN PLUM: MUC-3 Test Results and Analysis

Descriptive Note:

Conference paper

Corporate Author:


Report Date:


Pagination or Media Count:



Perhaps the most important facts about our participation in MUC-3 reflect our starting point and goals. In March, 1990, we initiated a pilot study on the feasibility and impact of applying statistical algorithms in natural language processing. The experiments were concluded in March, 1991 and lead us to believe that statistical approaches can effectively improve knowledge-based approaches Weishedel, et al., 1991a, Weischedel, Meteer, and Schwartz, 1991. Due to nature of that effort, we had focused on many well-defined algorithm experiments. We did not have a complete message processing system nor was the pilot study designed to create an application system. For the Phase I evaluation, we supplied a module to New York University. At the time of the Phase I Workshop 12-14 February 1991 we decided to participate in MUC with our own entry. The Phase I Workshop provided invaluable insight into what other sites were finding successful in this particular application. On 25 February, we started an intense effort not just to be evaluated on the FBIS articles, but also to create essential components e.g., discourse component and template generator and to integrate all components into a complete message processing system. Although the timing of the Phase II test 6-12 May was hardly ideal for evaluating our sites capabilities, it was ideally timed to serve as a benchmark prior to starting a four year plan for research and development in message understanding. Because of this, we were determined to try alternatives that we believed would be different than those employed by other groups, wherever time permitted. These are covered in the next section.Our results were quite positive, given these circumstances. Our max-tradeoff version achieved 45 recall and 52 precision with 22 overgenerating See Figure 2. PLUM can be run in several modes, trading off recall versus precision and overgeneration.

Subject Categories:

  • Information Science
  • Linguistics
  • Cybernetics

Distribution Statement: