Glossa Psycholinguistics

3.1 Participants and materials

100 German native speakers below the age of 50 (29 female, 71 male; mean age 30) were recruited on Prolific (https://www.prolific.co; Palan & Schitter, 2018). They were paid £5 for their participation. Additional prescreening was applied such that participants were required to have successfully completed 80% of previous experiments they had signed up for on Prolific.

Two sets of garden-path sentences each were constructed. For each sentence, both an early disambiguation version and a late disambiguation (garden-path) version were created. The first set contained the well-known NP coordination/S coordination ambiguity that exists in both German and English. Speakers initially prefer the NP-coordination analysis in these structures, and are garden-pathed when the structure is disambiguated towards sentential coordination (e.g., Frazier, 1987). An example sentence is shown in (2). Diamonds (⋄) indicate chunk boundaries. In the early disambiguation condition, the NP the rebel is masculine and carries unambiguous nominative case marking, which identifies it as the subject of a new clause. By contrast, in the late disambiguation condition, the NP is feminine and its case marking is ambiguous between nominative and accusative, thus allowing for an initial NP coordination misanalysis. Reanalysis should occur at guarded, which needs a subject.

The second set of sentences contained an ambiguity that does not exist in English, namely a subject-object ambiguity based on syncretism between nominative and accusative case for feminine noun phrases. German native speakers preferably analyze clause-initial case-ambiguous noun phrases as nominative-marked subjects and experience a garden path when the clause is disambiguated towards object-initial word order through number agreement on the verb (e.g., Hemforth, 1993; Meng & Bader, 2000a; Paape, Hemforth, & Vasishth, 2018). An example sentence is shown in (3). In the early disambiguation condition, the NP the animal keeper is masculine and overtly marked for accusative case, indicating an OVS structure. By contrast, in the late disambiguation condition, the NP is feminine and its case marking is ambiguous between nominative and accusative, allowing for an initial SVO misanalysis. Reanalysis should occur at sniffed (at) and licked, as the verbs are marked for plural and the would-be subject is singular.

For both sentence types, selective reanalysis as proposed by Frazier and Rayner (1982) predicts regressions from the verb region to the NP region. This prediction arises because the initial incorrect attachment of the noun phrases in the late-disambiguation (garden-path) conditions constitutes the error that reanalysis should aim to selectively correct. We have no specific prediction as to which of the two constructions should be more difficult to reanalyze. The main aim of the study is not to compare the two constructions, but to test the generality of selective reanalysis. Corpus data collected by Hoeks, Hendriks, Vonk, Brown, and Hagoort (2006) for the NP coordination versus S coordination in Dutch, which is typologically very close to German, and by Bader and Koukoulioti (2018) for German SVO versus OVS word order indicate quite similar frequency ratios of about 4.5/1 in favor of NP coordination and SVO word order, respectively. Nevertheless, the two structures may differ in their diagnosis and revision costs: Bader (2000) has argued that non-canonical OVS structures such as (3) are especially difficult to reanalyze compared to SVO structures, as required for the S coordination analysis in (2). Furthermore, Meng and Bader (2000a) have argued that the unexpected number agreement on the verb in (3) is not a particularly effective cue for reanalysis, because the morphological mismatch does not directly signal what the correct structure should be. Given these observations, it is possible that subject-object sentences create a stronger garden-path effect than coordination sentences.³

During the experiment, each subject read a total of 32 garden-path sentences drawn from a set of 64 via a Latin-squares procedure. Due to a classification error, there were 31 coordination ambiguity sentences and 33 subject-object ambiguity sentences. Fillers consisted of 20 sentences with varied structures, in addition to a set of 64 “memory” sentences, which all contained long-distance anaphoric references, e.g, “Johann had been given three five-Euro bills by his dad and a twenty-Euro bill by his mom to go shopping in the city. Unfortunately, he lost the money that his dad had given him on the bus”. Each subject read 32 of these memory sentences, which were designed to encourage rereading, resulting in a total of 84 sentences.

The garden-path stimuli were designed “with variability in mind” (Yarkoni, 2019): In order to increase the generalizability of the findings to a larger variety of natural language contexts, varying types of modifiers were used (relative clauses, prepositional phrases) and the absolute position of the critical regions as well as the distances between regions were varied between items. Additionally, some items consisted of multiple sentences while others consisted of a single complex sentence. Sentence content was varied to maximize engagement, and each stimulus was designed as a mini-discourse telling a coherent story. The full list of garden-path stimuli with their accompanying comprehension questions is given in the online appendix at https://osf.io/j8cbh.

3.2 Procedure and data analysis

The experiment was run on the Ibex farm Drummond (2013). Participants were instructed to read the sentences silently and answer the accompanying comprehension questions. They were told to press the right arrow key to move forward through the sentence and to press the left arrow key to move backward through the sentence. They were also informed that they could press the ESC key at any point to return directly to the beginning of the sentence, and the CTRL key to continue directly to the comprehension question.

A comprehension question was asked after every trial. Comprehension questions were detailed and targeted various aspects of the sentence, including the critical dependency in garden-path sentences in a small minority of cases, e.g., “How much money did Johann lose?”, “Where did the visitors see the animal keeper?”, “Did the opposition hide someone?”. Half of all comprehension questions were yes/no questions while the other half were multiple-choice questions with three possible answers each. Participants were instructed not to draw inferences beyond the literal content of the sentences when answering the questions.

Participants were told that there would be an anonymous high-score list, and that the ten highest scorers out of 100 would be paid an additional bonus of £9 each. They were told that they would gain 2 points for each correct answer to a comprehension question and that they would automatically lose 1 point every two minutes until the end of the experiment. After giving informed consent, participants completed two unscored practice trials, after which they were reminded of the scoring procedure and the function of each keyboard key. Afterwards, the main experiment began.

Due to their parallel structure, both kinds of garden-path sentences were analyzed together. Trials in which any single-region reading time was longer than 5000 ms were removed, resulting in a loss of 2% of the data. We chose right-bounded reading time (RBRT), regressive rereading time (RRT), and the proportion of first-pass regressions from a region as our measures of interest. Right-bounded reading time includes all reading times on the region prior to moving on to the next region; it is thus composed of first-pass reading time and progressive rereading time, that is, time spent rereading the region after having made a regression but before moving on to the right. By contrast, regressive rereading time includes all reading times on the region after having moved on to the right; it includes regressive as well as progressive rereading occurring after this point. Garden-pathing in the late disambiguation conditions compared to the early disambiguation conditions should lead to increased right-bounded reading times and possibly more first-pass regressions at the disambiguating verb region. Under the selective reanalysis hypothesis, regressive rereading times at the ambiguous NP region should be increased in the late disambiguation conditions.

All regions with RBRT and/or RRT below 150 ms were removed from the respective analyses, under the rationale that these very short RTs are the BSPR equivalent of skipping a region. This procedure resulted in the removal of 1% of the data for RBRT, and of about 80% of the data for RRT, mainly due to trials with RRT = 0 ms (that is, absence of any regressive rereading) being removed. Note that this means that the RRT analysis quantitatively compares the time spent rereading in the 20% of trials in which rereading above 150 ms actually occurred within the window of analysis. It does not capture differences between conditions in whether any rereading occurred at all. This aspect of the data is covered by the scanpath analysis presented below.

We defined a window of analysis ranging from the early disambiguation region – that is, the noun phrase at the beginning of the ambiguous region in the garden-path conditions – to two regions after the disambiguating verb.⁴ Linear mixed-effects models (LMMs) with lognormal likelihoods were fitted to individual regions using the brms package for Bayesian inference (Bürkner, 2017, 2018) in R (R Core Team, 2020). The models included the sum-coded predictors garden-path type (–1 for coordination, 1 for subject-object) and disambiguation (–1 for early, 1 for late). The full analysis code, including prior specifications, is given in the online appendix.

3.3 Reading measures: Descriptive results and LMM analysis

Figure 1 shows log-transformed RBRT and RRT by region. Subject-object sentences were read more slowly than coordination sentences across regions, conditions, and measures, likely due to the different clause structures, but we focus on the effects of disambiguation and its interactions. We report percentile-based 95% credible intervals of effects for which 95% of the posterior probability are above or below zero. Note that this is merely a convention we adopt for reporting effects for which there is some support in the data, and does not correspond to a null-hypothesis significance test. When interpreting the results, the reader should consider the width of the credible intervals, whether a given interval includes zero, and whether the effect replicates (see below). The full range of results is given in the online appendix.

Figure 1

Reading measures for Experiment 1. Due to different region lengths, log reading measures have been residualized against garden-path type and region ID to improve visibility. RBRT = Right-bounded reading time. RRT = Regressive rereading time. p (Regression) = Proportion of first-pass regressions.

3.4 Scanpath analysis

Scanpaths were analyzed using the Scasim algorithm developed by von der Malsburg and Vasishth (2011), which was designed for eye-tracking data. Scasim computes scanpath dissimilarities based on both fixation locations and fixation durations: If scanpaths have fixations in different locations and/or fixations of different durations in the same location, they are more dissimilar than scanpaths with fixations matching in both location and duration. The scanpath analysis complements the analysis of reading measures in multiple ways: First, it is holistic, in the sense that the entire trial is always considered instead of a single region. Second, it can capture differences in rereading probabilities that were not considered in the region-based analysis. Third, by dividing the scanpath similarities by the total duration of the trial, we obtain a measure of how attention was distributed across different regions of the sentence (see below).

In order to imitate eye-tracking data as closely as possible, we created x- and y-coordinates for each region of interest by setting y to a constant value n and x to n multiplied by the region number, thus simulating horizontal movement by increments of n. Here, we set n to 50. Region-wise reading times were used as fixation durations. For the scanpath analysis, reading times below 150 ms (“skips”) were retained, as they are an integral part of the scanpath’s overall shape. The reading times were log-transformed and residualized against region length, age of the participant, and the absolute position of the region within the sentence, including random effects by subject and by item. All effects had t-values >2. The residualization procedure is necessary because Scasim is blind to hierarchical structure in the data: Large differences between “fixations” can be due to extraneous factors, such as individual differences in reading speed and position-related speedups (e.g., Demberg & Keller, 2008; Ferreira & Henderson, 1993), lowering the chance of identifying the systematic reading patterns we are interested in.

In order to make scanpaths comparable across sentences, only the following regions were considered: the beginning of the sentence, the NP (early disambiguation) region, the immediately following modifier region(s), the verb (late disambiguation) region, the following verb+1 region, and the end of the sentence.⁵ Any “fixations” prior to the first visit to the NP region were discarded. The left panel of Figure 2 shows the critical portion of the scanpath predicted by selective reanalysis: When the disambiguating verb is encountered, a series of regressions should be triggered that targets the NP, quickly crossing over the modifier. After reanalysis has occurred, forward reading should resume, presumably without additional processing of subsequent regions. The right panel of Figure 2 shows another possible scanpath that would also show some indication of selectivity: The participant reads the entire sentence, then quickly returns to the beginning of the sentence, continues forward, carries out reanalysis at the NP, and then again proceeds forward. This pattern could be called “selective forward reanalysis”. By contrast, a more unselective rereading pattern would show approximately equal amounts of time spent in each region. Due to its sensitivity to differences in region-wise reading times, the scanpath analysis should be able to distinguish the two patterns.

Figure 2

Left: Partial scanpath predicted by selective reanalysis. Right: Hypothetical “selective forward reanalysis” scanpath. Note the increased rereading times at the NP region in both scanpaths. The y-axis shows the time spent in each region as a proportion of the total duration of the trial.

Scasim was set to normalize to total scanpath durations to obtain similarity per unit of time, as suggested by von der Malsburg and Vasishth (2011). The resulting similarity measure can be interpreted as the difference in the amount of attention paid to each region as a proportion of the duration of the entire trial. The matrix of pairwise scanpath similarities produced by Scasim was transformed into a two-dimensional Euclidean map using nonmetric multidimensional scaling as implemented in the vegan package (Oksanen et al., 2020). Given that multidimensional scaling algorithms are nondetermistic and may converge to different local stress optima,⁶ we repeated the procedure several times and chose the configuration with the lowest stress and the best interpretability (Borg & Mair, 2017). Mixture-based clustering was applied to the result using the mclust package (Scrucca, Fop, Murphy, & Raftery, 2016), and the number of clusters was reduced using the entropy criterion of Baudry, Raftery, Celeux, Lo, and Gottardo (2010).

The two dimensions of the resulting scanpath space can be roughly interpreted as “amount of rereading” and “location of regressions”. The scanpath map had a stress value of 11%. As shown in Figure 3, five clusters emerged: A large cluster of scanpaths with extensive rereading but no selective pattern (cluster 1, “extensive rereading”), two clusters of scanpaths with no rereading at all (clusters 2 and 3, “no rereading”), a cluster with single-region regressions into the NP region (cluster 4, “mod-to-NP”), and finally a cluster with single-region regressions from the verb region (cluster 5, “verb-to-mod”). Figure 4 shows example scanpaths for each of the clusters.

Figure 3

Scanpath map from Experiment 1. Note that the scanpaths in all clusters except cluster 1 (“extensive rereading”) are very uniform, which leads to overplotting. The rug plots in the margins show the densities along the two dimensions of the scanpath space.

Figure 4

Three example scanpaths for each of the five clusters from Experiment 1. The regions are as follows: 1 – Beginning of sentence, 2 — (removed), 3 – NP (early disambiguation), 4 – Post-NP, 5 – Verb (late disambiguation), 6 – Post-verb, 7 – (removed), 8 – End of sentence.

For each cluster, we fitted LMMs with Bernoulli likelihoods to the membership of the observed scanpaths (in cluster versus not in cluster). The models included condition (early disambiguation versus late disambiguation), garden-path type (coordination versus subject-object) and participants’ centered speed/accuracy point scores as fixed effects, as well as all two-way interactions with condition. Clusters 2 and 3 (“no rereading”) were collapsed for this analysis, as we could discern no meaningful difference between the patterns. Compared to early disambiguation, late disambiguation increased the probability of scanpaths falling into cluster 1, the “extensive rereading” cluster (= 4%, CrI: [1%, 8%]), and decreased the probability of scanpaths falling into clusters 2 and 3, the “no rereading” clusters (= –5%, CrI: [–9%, –1%]). For cluster 1 (“extensive rereading”), there was also an interaction between disambiguation and score (= 3%, CrI: [–1%, 6%]), such that high-scoring readers were especially likely to show extensive rereading in the late-disambiguation conditions (= 10%, CrI: [–2%, 22%]) compared to the early-disambiguation conditions ( = 3%, CrI: [–7%, 14%]).

Compared to coordination sentences, subject-object sentences had more scanpaths falling into cluster 4 (“mod-to-NP”), irrespective of disambiguation (= 2%, CrI: [0%, 3%]). Participants with higher scores were somewhat less likely to show this regression pattern (= –1%, CrI: [–2%, 0%]). There were no effects in evidence regarding membership in cluster 5 (“verb-to-mod”).

We investigated the relationship between cluster membership and question response accuracy by fitting LMMs to question response accuracies using cluster as a predictor. The predictor was treatment-coded, with the two “no rereading” clusters serving as the baseline. Across all clusters, participants’ mean question response accuracy was 83%. Across participants, accuracies ranged from 41% to 100% (1st quartile: 75%, 3rd quartile: 94%).⁷ Results of the analysis showed that scanpath membership in cluster 1 (“extensive rereading”) was associated with higher question response accuracy compared to clusters 2 and 3 (“no rereading”), irrespective of disambiguation ( = 4%, CrI: [1%, 6%]).

Lastly, in order to investigate how the varied rereading patterns within the “extensive rereading” cluster were affected by the experimental manipulations, we fitted LMMs with Gaussian likelihoods to both dimensions of the scanpath space of cluster 1. The models contained the categorical predictors disambiguation and garden-path type, as well as two continuous predictors: the total number of points scored by the participant in the experiment and the position of the trial in the experiment. Both continuous predictors were centered and scaled. The models also contained all two-way interactions with disambiguation.

Compared to early disambiguation, late disambiguation increased rereading on average ( = 0.05, CrI: [–0.01, 0.11]). Participants with higher scores also engaged in more rereading ( = 0.17, CrI: [0.06, 0.28]) and showed more varying regression locations ( = 0.05, CrI: [–0.01, 0.11]).

3.5 Discussion

Our results suggest that bidirectional self-paced reading can detect garden-path effects during right-bounded reading, that is, first-pass reading plus progressive rereading. Using scanpath analyses, recurring patterns can be identified in participant’s rereading strategies – or non-strategies.

Rereading was, by assumption, consciously controlled in this experiment. Under these conditions, we found no evidence of selective rereading, which should have manifested as a cluster of regressions from the verb region to the NP region in the garden-path conditions. Extensive, seemingly unselective rereading, however, occurred more often in the garden-path condition, and was tied to higher question response accuracy when compared against single-pass reading, across both garden-path and control sentences. There were also scanpath clusters with single, short regressions to an immediately preceding region. Such clusters were also observed by von der Malsburg and Vasishth (2011, 2013), who speculated that these regressions were tied to “checking” of word forms and/or corrections of premature forward saccades. Both explanations could also apply to our data. However, as the clusters were small and did not respond to the garden-path manipulation, we refrain from speculating about the precise function of these regressions and the possible role of awareness in their generation.

Our method also detected individual differences between participants. Participants with higher scores on the speed-accuracy measure across the entire experiment engaged in more rereading of late-disambiguated garden-path sentences, despite the fact that time spent rereading was penalized. It appears that these participants were able to generate a net benefit in comprehension accuracy through rereading, though there was no indication that this benefit was limited to garden-path sentences. We return to this point later.

We discuss the broader significance of the findings, especially with regard to selective reanalysis and the consciousness hypothesis, in section 5. We postpone the discussion because we were interested in whether our findings are replicable before judging their theoretical implications. We therefore conducted a direct replication study with new participants, using the same method and the same sentences, and ran the same statistical analyses on the new sample.

4.1 Participants and materials

A new sample of 100 German native speakers below the age of 50 was recruited (36 female, 64 male; mean age 30). Payment and prescreening were the same as in Experiment 1.

4.2 Procedure and data analysis

Procedure and data analysis were identical to Experiment 1. Again, trials in which any single-region reading time was longer than 5000 ms were removed, resulting in a loss of 1.6% of the data. Trials with reading measures below 150 ms (including RRT = 0 ms) were also removed, resulting in a loss of 1% the data for RBRT and 83% of the data for RRT.

4.3 Reading measures: Descriptive results and LMM analysis

Figure 5 shows log-transformed RBRT and RRT by region. As before, we focus on the effects of disambiguation and its interactions. The full range of results is given in the online appendix.

Figure 5

Reading measures for Experiment 2. Due to different region lengths, log reading measures have been residualized against garden-path type and region ID to improve visibility. RBRT = Right-bounded reading time. RRT = Regressive rereading time. p (Regression) = Proportion of first-pass regressions.

4.4 Scanpath analysis

As in Experiment 1, scanpaths clustered along the dimensions “amount of rereading” and “location of regressions”. The scanpath map had a stress value of 10%. As shown in Figure 6, five clusters emerged, most of which corresponded to the ones observed in Experiment 1: A cluster of scanpaths with extensive rereading (cluster 1, “extensive rereading”), a large cluster of scanpaths without any rereading (cluster 2, “no rereading”), a cluster with short regressions into the NP region (cluster 5, “mod-to-NP”), and a cluster with short regressions from the verb region to the preceding NP modifier region (cluster 4, “verb-to-mod”). A cluster that had not emerged in Experiment 1 showed short regressions into the verb region (cluster 3, “post-verb-to-verb”). Figure 7 shows example scanpaths for each of the clusters.

Figure 6

Scanpath map from Experiment 2. Note that the scanpaths in all clusters except cluster 1 (“extensive rereading”) are very uniform, which leads to overplotting. The rug plots in the margins show the densities along the two dimensions of the scanpath space.

Figure 7

Three example scanpaths for each of the five clusters from Experiment 2. The regions are as follows: 1 – Beginning of sentence, 2 — (removed), 3 – NP (early disambiguation), 4 – Post-NP, 5 – Verb (late disambiguation), 6 – Post-verb, 7 – (removed), 8 – End of sentence.

As in Experiment 1, garden-pathing increased the probability of membership in cluster 1, the “extensive rereading” cluster, though the effect was less pronounced ( = 1%, CrI: [–2%, 5%]). Unlike in Experiment 1, scanpaths of participants with higher scores were more likely to belong to this cluster ( = 14%, CrI: [5%, 25%]), and there was also an interaction between score and disambiguation ( = –4%, CrI: [–8%, –1%]): Cluster membership was more affected by score in the early-disambiguation conditions ( = 19%, CrI: [9%, 31%]) compared to the late-disambiguation conditions ( = 10%, CrI: [0%, 21%]). Participants with higher scores were less likely to show scanpaths belonging to cluster 2, the “no rereading” cluster ( = –19%, CrI: [–32%, –6%]). As in Experiment 1, scanpaths in subject-object sentences were more likely to fall into cluster 5 (“mod-to-NP”), irrespective of disambiguation ( = 2%, CrI: [0%, 3%]).

Participants’ mean question response accuracy was 82%. Across participants, accuracies ranged from 25% to 100% (1st quartile: 75%, 3rd quartile: 94%.⁸ As in Experiment 1, scanpath membership in cluster 1 (“extensive rereading”) was associated with higher question response accuracy compared to cluster 2 (“no rereading”), irrespective of disambiguation ( = 5%, CrI: [2%, 7%]). Furthermore, membership in cluster 4 (“verb-to-mod”) was associated with higher question response accuracy ( = 6%, CrI: [0%, 8%]). The latter effect is likely due to task demands, as comprehension questions often targeted information from the modifier. Scanpath membership in cluster 5 (“mod-to-NP”) increased accuracy in the early-disambiguation conditions ( = 6%, CrI: [0%, 8%]) but not in the late-disambiguation conditions ( = 0%, CrI: [–13%, 6%]). Given that early disambiguation occurred on the NP, this pattern could indicate successful “checking” of the NP’s disambiguating case feature.

As in Experiment 1, we analyzed the two scanpath dimensions within cluster 1 (“extensive rereading”). For the rereading dimension, late disambiguation sentences again showed more rereading compared to early disambiguation sentences ( = 0.06, CrI: [–0.02, 0.15]). As in Experiment 1, participants with higher scores engaged in more extensive rereading ( = 0.09, CrI: [0, 0.18]). There was also an interaction between disambiguation and trial number ( = 0.07, CrI: [0, 0.14]), such that rereading decreased with trial number for early-disambiguation sentences ( = –0.11, CrI: [–0.21, 0]) but not for late-disambiguation sentences ( = 0.03, CrI: [–0.08, 0.13]). This interaction may be spurious, as the opposite tendency was observed in Experiment 1. Unlike in Experiment 1, regression location did not seem to be affected by participant score ( = –0.02, CrI: [–0.06, 0.03]).

4.5 Discussion

The most relevant findings from Experiment 1 were replicated in Experiment 2. As in Experiment 1, the two garden-path types diverged at the early disambiguation region. For both garden-path types, there was a garden-path effect at the late disambiguation region in right-bounded reading times. Unlike in Experiment 1, the garden-path effect was also visible in first-pass regressions from the post-verbal region. Again, the scanpath analysis showed that garden-pathing led to more extensive rereading on average. There was again no indication of the rereading pattern predicted by selective reanalysis when rereading was manually controlled. However, extensive rereading was again tied to higher question response accuracy across conditions.

The early disambiguation penalty at the NP that we observed for unambiguous object-first sentences in both experiments is well-attested in German (Hemforth, 1993; Konieczny, 1996; Paape et al., 2018). The acceptability of OVS word order in German partly depends on contextual licensing conditions (Verhoeven & Temme, 2017; Weskott, Hörnig, Fanselow, & Kliegl, 2011) which may not always have been fully met in our items, despite prescreening for subjective acceptability by two native speakers. By contrast, the early ambiguity cost observed for coordination sentences may reflect competition between the NP coordination and S coordination analyses. However, earlier studies on the coordination ambiguity in Dutch did not show a competition effect (Hoeks et al., 2006; Hoeks, Vonk, & Schriefers, 2002). We suspect that the pattern in our studies may have been incurred by a clash between subjects’ underlying preference for a complex NP analysis and the chunking we imposed. As the potential complex NP was broken up (The opposition ⋄ hid ⋄ the wanted sympathizer ⋄ and ⋄ the rebel … ⋄ guarded …), subjects may have been pushed towards an S-coordination analysis. Given that our main conclusions do not depend on this pattern, we will not speculate on it in more depth.

We now turn to the implications of our findings for the consciousness hypothesis, as well as the more general implications for bidirectional self-paced reading as an experimental paradigm.

5.1 Implications for the consciousness hypothesis, and for selective reanalysis more generally

Our results do not lend credibility to the consciousness hypothesis, at least under the assumption that every key press in BSPR is the result of a conscious decision. We believe that this assumption is mostly warranted, especially for regressive key presses. For standard self-paced reading, it has been argued that subjects may start using a “tapping” strategy, advancing through the sentence mechanically at a fixed speed (Witzel et al., 2012). It is possible that the “no regression” clusters seen in our scanpath analyses correspond to such a strategy. Still, it is doubtful that readers would change direction without conscious awareness. When readers did decide to reread, regression patterns were varied and influenced by the experimental manipulations. Unselective rereading was more common in the late disambiguation conditions, that is, in the garden-path conditions. This matches earlier findings from eye tracking with Spanish sentences by von der Malsburg and Vasishth (2011, 2013), who also found more “unselective” rereading in the garden-path condition.

Our data do not indicate that readers focused on the ambiguous region of the sentence when they were garden-pathed: Across both experiments, there was no indication of increased regressive rereading times in the ambiguous NP region (= –9 ms, CrI: [–55 ms, 36 ms]), nor was there a scanpath cluster that would have indicated targeted revisits of the ambiguous NP region starting from the disambiguating verb region. The results thus do not lend support to the prediction that conscious “double takes” upon disambiguation coincide with selective reanalysis. By contrast, our findings are compatible with the view that selective reanalysis is not a general mechanism employed by the human sentence processor (Christianson et al., in prep.). However, given the assumed conscious nature of rereading the BSPR paradigm, it can also be argued that our results are compatible with the assumptions made by Frazier and Rayner (1982): Frazier and Rayner argued that conscious reanalysis is not selective, because it indicates a failure of automatic parsing routines. The absence of evidence for selective reanalysis in our studies could thus be taken to indicate a mixture of different types of trials: reanalysis was either easy enough to be carried out covertly (Lewis, 1998; von der Malsburg & Vasishth, 2013), or it was difficult, to an extent where both covert reanalysis and selective reanalysis failed, resulting in conscious, unselective rereading.

We do not know how often readers succeeded at reanalyzing our garden-path structures, as most comprehension questions did not target the relevant dependency. It is also possible that readers resorted to “good enough” processing in some proportion of trials, sticking to their incorrect initial interpretations (e.g., Christianson, Hollingworth, Halliwell, & Ferreira, 2001), or simply decided that the sentences were malformed, without even trying to engage in reanalysis (Fodor & Inoue, 2000; Meng & Bader, 2000b). These issues can be investigated in future work by using only comprehension questions that target the critical dependencies, or by using a grammaticality judgment task.

In our data sets, efficient readers – that is, readers who were fast and showed accurate comprehension across stimulus types – were more likely to engage in extensive, “unselective” rereading, and this type of rereading was tied to higher question response accuracy. While the pattern was not exclusive to garden-path sentences, it suggests that extensive rereading is, perhaps unsurprisingly, a useful strategy for information extraction. This may be especially true for sentences such as ours, which, in an attempt to increase naturalness, provided somewhat more context and plausibility than typical stimuli used in psycholinguistic research.

5.2 Generalizability of the results

Given the large number of participants, we could presumably have obtained narrower credible intervals for our effects, had we not chosen to increase the variability in our stimulus materials to aid generalizability. In our experiments, we obtained estimates of the same magnitude for the standard deviations of the slope adjustments to the garden-path effects by sentence and by subject (that is, by-item and by-subject “random slopes”). In a typical psycholinguistic experiment, the observed variability between sentences can be an order of magnitude smaller than the variability between subjects, because sentence structure is tightly controlled and real-life linguistic variation is eliminated (Yarkoni, 2019). This reduces noise and increases statistical power, but also reduces naturalness and generalizability. Our stimuli were, of course, still not natural in the sense of having been sampled from real-life language and occurring within a larger discourse (Hamilton & Huth, 2020), but presumably somewhat closer to participants’ everyday experience.

One important question related to generalizability is whether our findings extend to free reading. Despite its doubtful relationship with comprehension accuracy (Christianson et al., in prep.), “unselective” rereading may be the preferred strategy for handling garden paths in free reading (von der Malsburg & Vasishth, 2011, 2013), which matches our findings. Regressions in free reading are not fully automatic, as indicated by the fact that fewer regressions are made when previously read text is made unavailable for rereading through masking (Booth & Weger, 2013; Christianson et al., in prep.; Schotter et al., 2014; White, Lantz, & Paterson, 2017), presumably because there is little to gain from a regression. Furthermore, regression behavior during unmasked reading is influenced by task demands such as time pressure and the difficulty of comprehension questions (Godfroid, Loewen, et al., 2015; Weiss, Kretzschmar, Schlesewsky, Bornkessel-Schlesewsky, & Staub, 2018; Wotschack & Kliegl, 2013). As stated in the introduction, our task demands were tailored to favor selective rereading. Across our two experiments, subjects made at least one regression in garden-path sentences (early and late disambiguation) in about 47% of trials. The median number of regressions for these trials was 3, up to a maximum of 42. Participants jumped forward to the comprehension question in about 6% of trials and returned directly to the beginning of the sentence in about 5% of trials. The resulting variability in scanpath patterns is visualized in Figure 9, which shows a sample of scanpaths taken from cluster 1 (“extensive rereading”) of Experiment 2.

Figure 9

Example scanpaths for cluster 1 (“extensive rereading”) from Experiment 2. The regions are as follows: 1 – Beginning of sentence, 2 — (removed), 3 – NP (early disambiguation), 4 – Post-NP, 5 – Verb (late disambiguation), 6 – Post-verb, 7 – (removed), 8 – End of sentence.

Note that despite our use of the term “unselective” for this cluster of patterns, there are clear subpatterns showing rereading of the entire sentence (trial 2061), as well as trials showing the expected regression pattern of selective reanalysis (trial 2060), though without much indication of any prolonged rereading of the ambiguous NP region. In future work, we plan to examine whether comparable results can be obtained in an eye-tracking paradigm with the same task demands. Another open question that we plan to address is whether our findings from bidirectional self-paced reading generalize to other types of garden paths across languages. For instance, based on eye-tracking results, neither selective nor “unselective” rereading appears to increase comprehension accuracy for English NP/Z sentences (Since Jay always jogs a mile and a half seems …; Christianson et al., 2017, in prep.); this pattern may change when rereading is consciously controlled.

5.3 The need to consider conscious reading strategies

As a more general conclusion, we believe that research on syntactic ambiguity resolution and other psycholinguistic phenomena would benefit from closer engagement with the literature on metacognitive reading strategies that supplement automatic, lower-level reading skills (Afflerbach, Pearson, & Paris, 2008). Research in this domain often focuses on educational aspects, such as teaching efficient reading strategies to poor readers (Mokhtari & Reichard, 2002). Furthermore, the proposed strategies, such as taking notes or underlining parts of the text, usually target text-level comprehension, and are difficult to transfer to the sentence level and the modalities of a typical psycholinguistic experiment. Other strategies, however, may be highly relevant, such as the adjustment of reading speed according to text difficulty, use of contextual clues, active comprehension monitoring, and, most importantly in the present context, rereading (Mokhtari & Reichard, 2002). At the text level, strategic rereading improves comprehension, metacomprehension, and information recall (Millis & King, 2001; Rawson, Dunlosky, & Thiede, 2000; Stine-Morrow, Gagne, Morrow, & DeWall, 2004), but texts containing known triggers of processing difficulty such as negation may be exempt from this advantage (Margolin & Snyder, 2018). Given the results of Christianson et al., garden-path effects may be another example of this pattern.

Text-level reading strategies engage cognitive control mechanisms (Moss, Schunn, Schneider, McNamara, & VanLehn, 2011), which are also implicated in garden-path resolution (Novick, Hussey, Teubner-Rhodes, Harbison, & Bunting, 2014; Novick, Trueswell, & Thompson-Schill, 2005; Woodard, Pozzan, & Trueswell, 2016), as well as in the triggering of regressions during reading (Luke & Henderson, 2013). In this context, one interesting question for future research is whether encountering a disambiguating word in a garden-path sentence should be considered an explicit control trigger or an implicit control trigger. An example of an explicit control trigger would be a pre-trained visual or auditory cue that directly tells the participant to stop performing the current task (Logan, 1982). By contrast, implicit control triggers must be learned from the environment, such as the frequency of incongruent Stroop trials in a given experiment (Kunde et al., 2012). It would appear that disambiguating information in garden-path sentences is an implicit control trigger, given that the participant is not explicitly told to reanalyze the syntactic structure. The distinction is important with regard to selectivity, because implicit triggers of cognitive control have been argued to require awareness to be effective (Kunde et al., 2012). This would imply that controlled reanalysis – selective or unselective – cannot be carried unless the garden path is consciously registered. More generally, it remains to be seen if garden-path recovery is best characterized as a case of goal pursuit in the absence of conscious awareness (Custers & Aarts, 2010), or whether awareness has some role to play after all.