Accession Number:

ADA454807

Title:

Analysis of an Adaptive Control Scheme for a Partially Observed Controlled Markov Chain

Descriptive Note:

Technical research rept.

Corporate Author:

MARYLAND UNIV COLLEGE PARK SYSTEMS RESEARCH CENTER

Personal Author(s):

• Fernandez-Gaucherand, Emmanuel
• Arapostathis, Aristotle
• Marcus, Steven I.

Report Date:

1991-12-01

Pagination or Media Count:

16.0

Abstract:

The authors consider an adaptive finite state-controlled Markov chain with partial state information, motivated by a class of replacement problems. They present parameter estimation techniques based on the information available after actions that reset the state to a known value are taken. They prove that the parameter estimates converge w.p.1 to the true unknown parameter, under the feedback structure induced by a certainty equivalent adaptive policy. They also show that the adaptive policy is self-optimizing, in a long-run average sense, for any measurable sequence of parameter estimates converging w.p.1 to the true parameter.

Descriptors:

• *ADAPTIVE CONTROL SYSTEMS
• *STOCHASTIC PROCESSES
• *ESTIMATES
• *PARAMETERS
• *DYNAMIC PROGRAMMING
• ALGORITHMS
• FEEDBACK
• DIFFERENTIAL EQUATIONS
• REPLACEMENT
• CONVERGENCE
• COSTS
• OPTIMIZATION
• PROBABILITY DISTRIBUTION FUNCTIONS
• OBSERVATION

Subject Categories:

• Numerical Mathematics
• Statistics and Probability
• Operations Research

Distribution Statement:

APPROVED FOR PUBLIC RELEASE