DID YOU KNOW? DTIC has over 3.5 million final reports on DoD funded research, development, test, and evaluation activities available to our registered users. Click
HERE to register or log in.
Accession Number:
AD0641195
Title:
TWO SHORT NOTES ON MARKOV PROCESSES: I. A TEST FOR SUB-OPTIMAL ACTIONS IN MARKOVIAN DECISION PROBLEMS. II. AN INTRINSICALLY DETERMINED MARKOV CHAIN,
Descriptive Note:
Corporate Author:
WESTERN MANAGEMENT SCIENCE INST UNIV OF CALIFORNIA LOS ANGELES
Report Date:
1966-08-01
Pagination or Media Count:
13.0
Abstract:
In a Markovian decision problem choice of an action determines an immediate return and the probability of moving to the next state. It is desired to maximize the expected total of discounted future returns. If upper and lower bounds on the optimal expected return are available, a simple test is described which may show that certain actions are sub-optimal, permanently eliminating them from further consideration. This test may be incorporated into the dynamic programming routine for solving the decision problem. This was tried on Howards automobile replacement problem, using the upper and lower bounds described in A MModified dynamic programming method J. of Math. Anal. and Appl, 14 April, 1966. The amount of computation required by the dynamic programming routine was reduced, conservatively, by 75. Note I This paper describes a situation where there is a transient Markov chain with one absorbing state, such that the transition law of the process itself depends on the probability of the process being trapped in the absorbing state. Thus the process is defined implicitly, by its own behavior, so to speak. The existence and uniqueness of a transition law satisfying the given conditions is established. This result may give rise to additional ways of characterizing optimal policies, as is suggested by interpreting the result in terms of the behavior of a gambler. Note II
Distribution Statement:
APPROVED FOR PUBLIC RELEASE
