In this paper we review three kinds of proposed solutions to the catastrophic forgetting problem in neural networks. The solutions are based on reducing hidden unit overlap, rehearsal, and pseudorehearsal mechanisms. We compare the methods and identify some underlying similarities. We then briefly note some potential implications of the rehearsal/pseudorehearsal based methods, including their application to sequential learning tasks.

In this paper we present a novel neural network architecture that aims to combine the highly popular and successfulconvolutional neural network architecture with the learning mechanism of an unsupervised self-organizing map. The con-volutional self-organizing map (ConvSOM) is a hierarchical network consisting of several independent self-organizingmaps. It incorporates features associated with convolutional networks, such as weight sharing, spatial pooling, and hierar-chical abstraction, with the unsupervised, topographically organized self-organizing map. We will show that the resultingarchitecture performs poorly on the MNIST data set, but offers interesting avenues for further research.