Consider the problem of determining an optimal replacement policy for a piece of equipment which is subject to Markovian changes of state. The equipment is inspected at discrete time intervals and is classified to be in one of a finite set of states. There is an occupancy cost associated with each of the states. One of the states represents a failure state with an associated penalty cost. On failure the equipment is brought to the new state, by replacement at a cost. On inspection the choice is to replace the equipment or to let it deteriorate in a Markovian manner, the objective being to minimize the expected average cost per period over a long run operation. The principal result is to show that under a general aging property on the stochastic matrix of equipment deterioration and when the state occupancy cost is increasing as the states approach the failure state then only control limit rules are optimal.