On Stochastic Approximation

Stochastic approximation is concerned with schemes converging to some sought valuewhen, due to the stochastic nature of the problem, the observations involve errors. Theinteresting schemes are those which are self-correcting, that is, in which a mistake alwaystends to be wiped out in the limit, and in which the convergence to the desired value isof some specified nature, for example, it is mean-square convergence. The typical exampleof such a scheme is the original one of Robbins-Monro 7 for approximating, undersuitable conditions, the point where a regression function assumes a given value. Robbinsand Monro have proved mean-square convergence to the root Wolfowitz 8 showedthat under weaker assumptions there is still convergence in probability to the root andBlum 11 demonstrated that, under still weaker assumptions, there is not only convergencein probability but even convergence with probability 1. Kiefer and Wolfowitz 6have devised a method for approximating the point where the maximum of a regressionfunction occurs. They proved that under suitable conditions there is convergence in probabilityand Blum 1 has weakened somewhat the conditions and strengthened the conclusionto convergence with probability 1.