Skip to main content
eScholarship
Open Access Publications from the University of California

Learning to count without a counter : A case study of dynamics and activation landscapes in recurrent networks

Abstract

The broad context of this study is the investigation of the nature of computation in recurrent networks (RNs). The cur?rent study has two parts. The first is to show that a R N can solve a problem that we take to be of interest (a counting task), and the second is to use the solution as a platform for develop?ing a more general understanding of RN s as computational mechanisms. W e begin by presenting the empirical results of training RN s on the counting task. The task {a b ) is the sim?plest possible grammar that requires a PD A or counter A R N was trained to predict the deterministic elements in sequences of the form a"b" * where n=l to 12. After training, it general?ized to n=18. Contrary to our expectations, on analyzing the hidden unit dynamics, we find no evidence of units acting like counters. Instead, we find an oscillator W e then explore the possible range of behaviors of oscillators using iterated maps and in the second part of the paper we describe the use of iter?ated maps for understanding R N mechanisms in terms of "activation landscapes". This analysis leads to used an under?standing of the behavior of network generated in the simula?tion study.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View