Proceedings of the Annual Meeting of the Cognitive Science Society

Standard methods in deep learning fail to capture composi-tional or systematic structure in their training data, as shownby their inability to generalize outside of the training distribu-tion. However, human learners readily generalize in this way,e.g. by applying known grammatical rules to novel words. Theinductive biases that might underlie this powerful cognitive ca-pacity remain unclear. Inspired by work in cognitive sciencesuggesting a functional distinction between systems for syn-tactic and semantic processing, we implement a modificationto an existing deep learning architecture, imposing an analo-gous separation. The resulting architecture substantially out-performs standard recurrent networks on the SCAN dataset, acompositional generalization task, without any additional su-pervision. Our work suggests that separating syntactic fromsemantic learning may be a useful heuristic for capturing com-positional structure, and highlights the potential of using cog-nitive principles to inform inductive biases in deep learning.