DID YOU KNOW? DTIC has over 3.5 million final reports on DoD funded research, development, test, and evaluation activities available to our registered users. Click
HERE to register or log in.
Accession Number:
ADA281027
Title:
Adaptive Statistical Language Modeling; A Maximum Entropy Approach
Descriptive Note:
Doctoral thesis
Corporate Author:
CARNEGIE-MELLON UNIV PITTSBURGH PA DEPT OF COMPUTER SCIENCE
Report Date:
1994-04-19
Pagination or Media Count:
117.0
Abstract:
Language modeling is the attempt to characterize, capture and exploit regularities in natural language. In statistical language modeling, large amounts of text are used to automatically determine the models parameters. Language modeling is useful in automatic speech recognition, machine translation, and any other application that processes natural language with incomplete knowledge. In this thesis, I view language as an information source which emits a stream of symbols from a finite alphabet the vocabulary. The goal of language modeling is then to identify and exploit sources of information in the language stream, so as to minimize its perceived entropy. Most existing statistical language models exploit the immediate past only. To extract information from further back in the documents history, I use trigger pairs as the basic information bearing elements. This allows the model to adapt its expectations to the topic of discourse. Next, statistical evidence from many sources must be combined. Traditionally, linear interpolation and its variants have been used, but these are shown here to be seriously deficient. Instead, I apply the principle of Maximum Entropy ME. Each information source gives rise to a set of constraints, to be imposed on the combined estimate. The intersection of these constraints is the set of probability functions which are consistent with all the information sources. The function with the highest entropy within that set is the NE solution. Language modeling, Adaptive language modeling, Statistical language modeling, Maximum entropy, Speech recognition.
Distribution Statement:
APPROVED FOR PUBLIC RELEASE
