Multi-layer perceptrons (MLPs) can learn both regular and irregular items given sufficient interleaved training, but not from sequential presentation of items. McClelland, McNaughton and O'Reilly (1994) addressed this problem in their proposal that the hippocampus and neocortex (H/NC) form a two component memory system in which the hippocampus interleaves training of items to the neocortex so that it can develop structure without interference of later items on earlier ones. We have been studying such an interleaving system under the constraint of limiting the capacity of the training batch (analogous to a finite limit on the hippocampus). In previous simulations (Gray & Wiles, 1996) we demonstrated that a quasi-regular learning task trained with a recency rehearsal scheme did not suffer interference to a catastrophic level, but did suffer interference on irregular and similar regular items. The current study introduces a new rehearsal scheme in which items are retained in a finite training batch based on how well the MLP has learned them: Error rehearsal enabled the MLP to learn (1) a high proportion of the domain, (2) retention of both regular and irregular items from the initial training batch and (3) partial shielding of both regular and irregular items from later interference. The results demonstrate that although finite training batches can pose a problem for MLPs, an error rehearsal scheme can reduce interference on both regular and irregular items, even when they are no longer in the current training batch. Implications for the role of the hippocampus in interleaving items for the neocortex are discussed.
