Analyzing the Energy Landscapes of Distributed Winner-Take-All Networks
Technical rept. 15 Sep 1986-14 Sep 1991
CARNEGIE-MELLON UNIV PITTSBURGH PA ARTIFICIAL INTELLIGENCE AND PSYCHOLOGY PROJECT
Pagination or Media Count:
DCPS the Distributed Connectionist Production System is a neural network with complex dynamical properties. Visualizing the energy landscapes of some of its component modules leads to a better intuitive understanding of the model, and suggests ways in which its dynamics can be controlled in order to improve performance on difficult cases. Competition through mutual inhibition appears in a wide variety of network designs. This paper discusses a system with unusually complex competitive dynamics. The system is DCPS, the Distributed Connectionist Production System of Touretzky and Hinton 1988. DCPS is a Boltzmann machine composed of five modules, two of which, labeled Rule Space and Bind Space, are winner-take-all WTA networks. These modules interact via their effects on two attentional modules called clause spaces. Clause spaces are another type of competitive architecture based on mutual inhibition, but they do not produce WTA behavior. Both clause spaces provide evidential input to both WTA nets, but since connections are symmetric they also receive top-down guidance from the WTA nets. Thus, unlike most other competitive architectures, in DCPS the external input to a WTA net does not remain constant as its state evolves. Rather, the present output of the WTA net helps to determine which evidence will become visible in the clause spaces in the future. This dynamic attentional mechanism allows rule and bind spaces to work together even though they are not directly connected. Keywords Stochastic search.