Proceedings of the Annual Meeting of the Cognitive Science Society

Grammatical knowledge and how it is empirically tested are typically considered robust to the frequency of the lexical items used in the expressions. Complementary Learning Systems theory proposes that hippocampal episodic memory, which enables rapid encoding and retrieval of specific experiences, allows learners to leverage those experiences when processing rare patterns. We test the hypothesis that robustness to lexical frequency can arise via such an episodic memory mechanism by evaluating whether retrieval-augmented language models (specifically, k-nearest-neighbor language models that augment parametric neural networks with explicit instance storage), help close the lexical frequency gap in syntactic contrasts that vanilla language models exhibit. Using syntactic contrasts with frequency-stratified test items, we find that retrieval augmentation leads to improvements for test instances containing low-frequency lexical items, consistent with episodic memory compensating for weak parametric representations. This benefit is consistent across different syntactic phenomena and across models pretrained on child-realistic and large-scale data. Additionally, we show that structural information is critical for effective retrieval, whereas semantic information is only responsible for minor gains. While these are promising proof-of-concept results supporting our hypothesis, the frequency gap remains not fully closed. Based on our analyses, we posit preferential reweighting of retrieved instances, better representations and retrieval strategies of structural information, and flexible configurations of storage and retrieval as promising future directions for improving the implementation of episodic memory in language models.