PARTIALLY OBSERVABLE MARKOV PROCESSES.
Technical rept. no. 4
MASSACHUSETTS INST OF TECH CAMBRIDGE OPERATIONS RESEARCH CENTER
Pagination or Media Count:
A partially observable Markov process is a model of a discrete time dynamic system which takes into account the effects of imperfect observations and of random system be havior. The model consists of an underlying Markov process with state vector Xn. Direct observations of Xn are not possible, but a vector Zn is observed. The observation Zn is related to the state Xn by a known probability density function. This model is useful in the analysis of a very large class of sequential decision problems. It was shown that a partially observable Markov process is conveniently analyzed by the introduction of the probability density function. This density function was shown to have certain characteristic iterative properties and is referred to as the statistical state of the system. The application of the theory of partially observable Markov processes to the problems of estimation, prediction and smoothing is straightforward. When a general terminal control problem is considered, however, the notion of minimum expected cost turns out to be ambiguous. The concepts of a priori and a posteriori control were introduced to reslove this confusion. Author