Proceedings of the Annual Meeting of the Cognitive Science Society

The disambiguation of syntactically ambiguous sentences canlead to reading difficulty, often referred to as a garden path ef-fect. The surprisal hypothesis suggests that this difficulty canbe accounted for using word predictability. We tested this hy-pothesis using predictability estimates derived from two fam-ilies of language models: grammar-based models, which ex-plicitly encode the syntax of the language; and recurrent neuralnetwork (RNN) models, which do not. Both classes of mod-els correctly predicted increased difficulty in ambiguous sen-tences compared to controls, suggesting that the syntactic rep-resentations induced by RNNs are sufficient for this purpose.At the same time, surprisal estimates derived from all mod-els systematically underestimated the magnitude of the effect,and failed to predict the difference between easier (NP/S) andharder (NP/Z) ambiguities. This suggests that it may not bepossible to reduce garden path effects to predictability