The number of the subject in English must match the num-ber of the corresponding verb (dog runs but dogs run). Yetin real-time language production and comprehension, speak-ers often mistakenly compute agreement between the verb anda grammatically irrelevant non-subject noun phrase instead.This phenomenon, referred to as agreement attraction, is mod-ulated by a wide range of factors; any complete computationalmodel of grammatical planning and comprehension would beexpected to derive this rich empirical picture. Recent develop-ments in Natural Language Processing have shown that neuralnetworks trained only on word-prediction over large corporaare capable of capturing subject-verb agreement dependen-cies to a significant extent, but with occasional errors. In thispaper, we evaluate the potential of such neural word predic-tion models as a foundation for a cognitive model of real-timegrammatical processing. We use LSTMs, a common sequenceprediction model used to model language, to simulate six ex-periments taken from the agreement attraction literature. TheLSTMs captured the critical human behavior in three out of thesix experiments, indicating that (1) some agreement attractionphenomena can be captured by a generic sequence process-ing model, but (2) capturing the other phenomena may requiremodels with more language-specific mechanisms.

One of the key features of natural languages is that they exhibit long-distance filler-gap dependencies (FGDs): In the sentence `What do you think the pilot sent __?' the wh-filler what is interpreted as the object of the verb sent across multiple words. The ability to establish FGDs is thought to require hierarchical syntactic structure. However, recent research suggests that recurrent neural networks (RNNs) without specific hierarchical bias can learn complex generalizations about wh-questions in English from raw text data (Wilcox et al. 2018; 2019). Across two experiments, we probe the generality of this result by testing whether a long short-term memory (LSTM) RNN model can learn basic generalizations about FGDs in Norwegian. Testing Norwegian allows us to assess whether previous results were due to distributional statistics of the English input or whether models can extract similar generalizations in languages with different syntactic distributions. We also test the model's performance on two different FGDs: wh-questions and relative clauses, allowing us to determine if the model learns abstract generalizations about FGDs that extend beyond a single construction type. Results from Experiment 1 suggest that the model expects fillers to be paired with gaps and that this expectation generalizes across different syntactic positions. Results from Experiment 2 suggest that the model's expectations are largely unaffected by the increased linear distance between the filler and the gap. Our findings provide support for the conclusion that LSTM RNN's ability to learn basic generalizations about FGDs is robust across dependency type and language.