A novel connectionist model of sentence production is pre-sented, which employs rich situation model representationsoriginally proposed for modeling systematicity in comprehen-sion (Frank, Haselager, & van Rooij, 2009). The high overallperformance of our model demonstrates that such represen-tations are not only suitable for comprehension, but also formodeling language production. Further, the model is able toproduce novel encodings (active vs. passive) for a particularsemantics, as well as generate such encodings for previouslyunseen situations, thus demonstrating both syntactic and se-mantic systematicity. Our results provide yet further evidencethat such connectionist approaches can achieve systematicity,in production as well as comprehension.

Recent models of language have eliminated syntactic-semanticdividing lines. We explore the psycholinguistic implicationsof this development by comparing different types of sentenceembeddings in their ability to encode syntactic constructions.Our study uses contrasting sentence structures known to causesyntactic priming effects, that is, the tendency in humans to re-peat sentence structures after recent exposure. We comparehow syntactic alternatives are captured by sentence embed-dings produced by a neural language model (BERT) or by thecomposition of word embeddings (BEAGLE, HHM, GloVe).Dative double object vs. prepositional object and active vs.passive sentences are separable in the high-dimensional spaceof the sentence embeddings and can be classified with a highdegree of accuracy. The results lend empirical support to themodern, computational, integrated accounts of semantics andsyntax, and they shed light on the information stored at differ-ent layers in deep language models such as BERT.