Stochastic Gradient Algorithms for Searching Multidimensional Multimodal Surfaces.

Certain optimization problems can be reduced to the form given a criterion function of vector W dependent upon a set of scalar adjustments acting as components of a vector vector W find a vector vector W such that hvector W or h vector W for all admissible vector W. It is often helpful to consider hvector W a surface defined on a multidimensional vector space. The stochastic gradient approach to the problem of searching a general multidimensional surface for a minimum is based upon the fact that an unbiased gradient estimate for a steepest descent algorithm can be obtained easily from a measurement of the surface at a random displacement of the basepoint. Unlike other steepest descent methods, this method does not necessarily impose the restriction that the perturbation be kept small to obtain an accurate gradient estimate. Author