Skip to main content
eScholarship
Open Access Publications from the University of California

Comparing Gated and Simple Recurrent Neural Network Architectures as Modelsof Human Sentence Processing

Abstract

The Simple Recurrent Network (SRN) has a long tradition incognitive models of language processing. More recently, gatedrecurrent networks have been proposed that often outperformthe SRN on natural language processing tasks. Here, we in-vestigate whether two types of gated networks perform betteras cognitive models of sentence reading than SRNs, beyondtheir advantage as language models. This will reveal whetherthe filtering mechanism implemented in gated networks corre-sponds to an aspect of human sentence processing. We traina series of language models differing only in the cell types oftheir recurrent layers. We then compute word surprisal valuesfor stimuli used in self-paced reading, eye-tracking, and elec-troencephalography experiments, and quantify the surprisalvalues’ fit to experimental measures that indicate human sen-tence reading effort. While the gated networks provide betterlanguage models, they do not outperform their SRN counter-part as cognitive models when language model quality is equalacross network types. Our results suggest that the differentarchitectures are equally valid as models of human sentenceprocessing.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View