The TACITUS System: The MUC-3 Experience
SRI INTERNATIONAL MENLO PARK CA ARTIFICIAL INTELLIGENCE CENTER
Pagination or Media Count:
SRI International has been engaged in research on text understanding for a number of years. The Naval Ocean Systems Center NOSC has sponsored three workshops in recent years for evaluating text understanding systems. SRI participated in the first Message Understanding Conference MUC-1 in June 1987 as an observer, and subsequently as a participant. Our system was evaluated in the second and third workshops, MUC-2 and MUC-3. For MUC-2, the task that the systems had to perform was to extract information for database entries saying who did what to whom, when, where, and with what result. The application domain for MUC-3 was news articles on terrorist activities in Latin America. The task was similar to that in MUC-2, though somewhat more information had to be extracted. The principal measures in the MUC-3 evaluation were recall and precision. Recall is the number of answers the system got right divided by the number of possible right answers. It measures how comprehensive the system is in its extraction of relevant information. Precision is the number of answers the system got right divided by the number of answers the system gave. It measures the systems accuracy. The system SRI used for these evaluations is called TACITUS. TACITUS is a system for interpreting natural language texts that has been under development since 1985. It has a preprocessor and postprocessor currently tailored to the MUC-3 application. It performs a syntactic analysis of the sentences in the text, using a fairly complete grammar of English, producing a logical form in first-order predicate calculus. Pragmatics problems are solved by abductive inference in a pragmatics, or interpretation, component.
- Information Science
- Computer Programming and Software