Proceedings of the Annual Meeting of the Cognitive Science Society

Recurrent neural networks (RNNs) remain central to cognitive modeling, despite the rise of transformers, because their recurrent dynamics are relatively cognitively plausible. But many questions remain about the strengths and weaknesses of RNNs as learning systems and cognitive models. We compare two architectures: Elman networks with hidden-layer recurrence and Jordan networks with output-layer recurrence. Using carefully constructed artificial languages, we show that Elman networks excel at learning long-range dependencies, especially when intervening elements provide no predictive cues. Jordan networks perform equally well when adjacent elements correlate with target outputs or when sequence-level error signals are available. However, Jordan networks falter in particular with item-level error feedback (as opposed to sequence-level error feedback), particularly for nonlinear sequential relations such as XOR. These results clarify how recurrence placement shapes sequence learning, suggest links to cognitive and neural processing, and highlight the representational advantages of hidden-layer recurrence over simple output chaining.