Cluster analysis has been successfully applied to the problem understanding hidden unit representations in both feed-forward and recurrent neural networks. While the topological properties of feed-forward networks may support the use of cluster analysis, the results described within this paper suggest that applications to recurrent networks are not justified. This paper illustrates how clustering fails to provide useful insights into the underlying task-dependent information processing mechanism of recurrent networks. In this paper, I first demonstrate that randomly generated networks display a surprising amount of clustering before training. Then I explain that the clustering structure emerges, not in response to the task training, but because of the volume-reducing iterated mappings that comprise the commonly used recurrent neural networks models.

Many researchers in AI and Cognitive Science believe that the information processing complexity of a mecha- nism is reflected in the complexity of a description of its behavior. In this paper, w e distinguish two types of com- plexity and demonstrate that neither one can be an objective property of the underlying physical system. A shift in the method or granularity of observation can cause a system's behavioral description to change in both the number of apparent states and the complexity class. These examples demonstrate h o w the act of obser- vation itself can suggest frivolous explanations of physi- cal phenomena, up to and including computation.

This paper discusses the problem of how to implement many-to-many, or multi-associative, mappings within connectionist models. Traditional symbolic approaches wield explicit representation of all alternatives via stored links, or implicitly through enumerative algorithms. Classical pattern association models ignore the issue of generating multiple outputs for a single input pattern, and while recent research on recurrent networks is promising, the field has not clearly focused upon multi-associativity as a goal. In this paper, we define multiassociative memory M M , and several possible variants, and discuss its utility in general cognitive modeling. W e extend sequential cascaded networks (Pollack 1987, 1990a) to fit the task, and perform several initial experiments which demonstrate the feasibility of the concept.

This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the badt-propagation technique. W e first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of bade propagation to initial weight configuration.

Cognitive scientists, AI researchers in particular, have long-recognized the enormous benefits of modularity (e.g., Simon, 1969), as well as the need for self-organization (Samuel, 1967) in creating artifacts whose complexity approaches that of human intelligence. And yet these two goals seem almost incompatible, since truly modular systems are usually designed, and systems that truly learn are inherently nonmodular and produce only simple behaviors. Our paper seeks to remedy this shortcoming by developing a new architecture of Additive A d i ^ v e Modules which we instantiate as Addam, a modular agent whose behavioral repertoire evolves as the complexity of the environment is increased.'