Glossa Psycholinguistics

1. Introduction

During reading, 10% to 15% of readers’ eye movements go against the forward flow of the text, that is, they are regressive (Rayner, 1998; Rayner et al., 2006). Regressions occur less frequently when previously read text is masked, suggesting that their purpose is to allow rereading (Booth and Weger, 2013; Schotter et al., 2014). The need for rereading can result from a failure to integrate the current word into the sentential context when it is highly unexpected or incongruous (e.g., Yan and Jaeger, 2020; Staub, 2011). When readers encounter an unexpected word (Since Jay always jogs a mile and a half seems …), this may lower their confidence in the current interpretation of the previous sentence context (Bicknell and Levy, 2011; Weiss, 2020; Levy, 2008; Levy et al., 2009), and they may launch a regression to either confirm or revise their current interpretation (e.g., Christianson et al., 2017; Frazier and Rayner, 1982). If the reader’s task is to judge a given sentence’s syntactic and/or semantic well-formedness, rereading may serve to confirm a first impression (Paape et al., 2021; Godfroid et al., 2015a), resulting in higher judgment accuracy compared to trials without rereading (Metzner et al., 2017).

A topic that has received comparatively little attention in psycholinguistic studies of regressions and rereading is the role of conscious awareness. Skilled reading is largely considered to be a highly automatic process (Reichle et al., 1998; Logan, 1997), and conscious awareness of processing difficulty, which may result in deliberate rereading, has mostly been studied in groups whose processing is presumably less automatic, that is, in non-native speakers (see Godfroid et al., 2015b for a review) and in children (e.g, Eilers et al., 2018).

For instance, in an eye-tracking study, Hessel et al. (2021) investigated how 9- to 10-year-olds respond to inconsistent discourse continuations such as kitten in (1).

Hessel et al. found that compared to a consistent discourse continuation (puppy), children were more likely to reread the word kitten, and also spent more time rereading it. Hessel et al. also found a correlation between self-reports of deliberate rereading strategies (“I go back to words and sentences that I have found difficult to understand when I first read them”) and observed rereading of the kitten/puppy region in the eye-tracking record, but no interaction with discourse consistency. Hessel et al. concluded that deliberate rereading may play only a minor role in dealing with such inconsistencies, which may often be resolved in a largely automatic fashion (Van den Broek and Helder, 2017).

An open question is whether evidence for deliberate rereading can be observed in adult native speakers, with a different type of linguistic manipulation that does not target discourse consistency, and/or by using different tasks and reading paradigms. The distinction between confirmatory and revisionary rereading (Christianson et al., 2017) is of particular interest in this context. In the sentences investigated by Hessel et al. (2021), children presumably engaged in confirmatory rereading, that is, they wanted to make sure that the sentence was indeed about a barking kitten. Revisionary rereading, on the other hand, has been argued to play a role in garden-path sentences such as (2):

Here, the noun phrase a mile and a half is likely to be initially misanalyzed as the direct object of the verb jogs as opposed to the subject of a new clause. Once the verb seems is read, the sentence is disambiguated towards an analysis in which jogs is used intransitively. This type of syntactic ambiguity is known as the NP/zero complement (NP/Z) ambiguity. The selective reanalysis hypothesis proposed by Frazier and Rayner (1982) states that when the disambiguating information arrives, readers will preferentially reread those parts of the sentence that they misanalyzed on the first pass. This type of rereading is revisionary in the sense that it coincides with syntactic reanalysis. For (2), the prediction is that readers will return to the verb jogs and/or to the problematic noun phrase a mile and a half in order to find an alternative syntactic analysis, namely one in which a mile and a half is the subject of seems rather than the object of jogs.

Frazier and Rayner assume that this overt, selective process of syntactic reanalysis is the parser’s default recovery strategy after being garden-pathed, and occurs in the absence of conscious awareness. They argue that because conscious awareness of garden-pathing is rare, it cannot reflect the parser’s “fundamental response to the pervasive temporary ambiguity of natural languages” (p. 179), but rather take awareness as an index of failure to selectively reanalyze the sentence. Frazier and Rayner found some evidence of selective reanalysis in their eye-tracking study, but the evidence from later studies investigating selective reanalysis has been much more mixed (Mitchell et al., 2008; Meseguer et al., 2002; von der Malsburg and Vasishth, 2011, 2013; Schotter et al., 2014; Christianson et al., 2017, in prep.). Importantly, Christianson et al. (2017) and Christianson et al. (in prep.) found no relationship between rereading of garden-path sentences and comprehension accuracy: Participants often misinterpret a sentence like (2) to mean that Jay always jogs a mile and a half even after rereading it. This suggests that the purpose of rereading in garden-path sentences may be confirmatory rather than revisionary, similar to the puppy/kitten sentences of Hessel et al. (2021).

As a possible explanation for the mixed results in the literature, Paape and Vasishth (2022) suggested that selective reanalysis may be tied to conscious awareness of the garden path, contra Frazier and Rayner (1982). Depending on the syntactic construction, garden paths may be mild or strong, and may or may not rise to awareness. The NP/Z garden path in particular is relatively strong and may cause readers to consciously experience parsing failure. In such cases, it is possible that selective, truly revisionary rereading is observed, indexing a deliberate attempt to find a grammatical structure. Such deliberate rereading can be triggered by naturally-occurring newspaper headlines such as Google’s computer might betters translation tool (Novick et al., 2014), which often cause the reader to do a double take and reread the sentence. Subjectively, in such difficult sentences, a switch occurs from automatic processing to a more deliberate, metacognitive process of reanalysis in which the comprehender actively tries to figure out where they went wrong (Marcus 1980; Lewis 1998; Gibson 1991).¹

A variety of scientific tools exist to study deliberate reading strategies, including self-reports and “thinking aloud” paradigms, which have the advantage of tracking strategy use as it occurs (Cromley and Azevedo, 2006). As an additional tool that requires neither introspection nor verbalization on part of participants, Paape and Vasishth (2022) proposed the bidirectional self-paced reading (BSPR) paradigm. BSPR is an extension of “moving-window” self-paced reading (SPR; Mitchell and Green, 1978; Aaronson and Scarborough, 1976), which does not allow regressions. By contrast, in BSPR, participants can move forward or backward through the sentence by pressing specific keyboard keys, for instance the right and left arrow keys. By pressing one of two additional keys, they can also return directly to the beginning of the sentence, or proceed directly to the end-of-trial task from any point in the sentence.² Paape and Vasishth (2022) argue that while regressions and rereading in eye tracking may be purely automatic to some extent, regressions in BSPR are always conscious, because the participant needs to make a deliberate decision to press the “back” key.

To test their hypothesis that selective reanalysis is connected to conscious awareness, Paape and Vasishth (2022) applied the BSPR paradigm to German garden-path sentences. At the disambiguating region, they found reading slowdowns consistent with garden-path effects in right-bounded reading times (that is, before readers read past the region), but no indication of selective rereading of earlier material. Instead, participants engaged in a variety of rereading strategies, including rereading of the entire sentence (see also von der Malsburg and Vasishth, 2011, 2013; Christianson et al., in prep.). Given the absence of evidence for selective reanalysis in a conscious reading setting, there is reason to doubt the assumption that selective rereading is tied to conscious awareness. However, a possible reason for the absence of selective rereading in Paape and Vasishth’s study is that the garden-path effects in the German sentences they used may be milder in comparison to the NP/Z ambiguity studied by Frazier and Rayner (1982).³ If the garden-path effect in the German materials was not strong enough to rise to consciousness, one would not expect conscious rereading to be triggered.

To further explore this issue, we conduct a BSPR study to investigate deliberate, selective rereading in NP/Z sentences, in which the garden path is often consciously registered. We also directly compare NP/Z sentences to the discourse inconsistency sentences used by Hessel et al. (2021), as well as to two additional constructions from the psycholinguistic literature. In order to distinguish confirmatory rereading in the BSPR task from revisionary rereading, we use end-of-trial acceptability judgments. This offline measure allows us to distinguish selective rereading that is purely confirmatory from selective reanalysis, which is revisionary by the definition of Frazier and Rayner (1982). Specifically, we ask participants to reject sentences if they believe they are ungrammatical or nonsensical. Garden-path sentences are sometimes rejected as ungrammatical, presumably because readers cannot find the correct syntactic analysis (Ferreira and Henderson, 1991; Warner and Glass, 1987). If selective rereading indexes selective reanalysis, and if reanalysis entails the computation of a grammatically well-formed structure for the sentence, participants should be more likely to judge garden-path sentences as grammatical in trials with selective rereading. By contrast, if readers merely reread to confirm their initial analysis, offline judgments should not vary depending on whether rereading occurred or not: Either participants succeed at reanalyzing the garden-path structure covertly, without regressing (Lewis, 1998), and then reread to confirm the reanalyzed structure, or rereading simply confirms the perceived ungrammaticality of the sentence.

Besides NP/Z garden-path sentences and discourse inconsistency sentences, our study uses sentences with a main clause/reduced relative clause (RRC) ambiguity (the lawyer sent by the governor …), which causes milder garden-path effects than the NP/Z ambiguity. The fourth construction used in the study consists of reflexive sentences with grammatically unavailable but gender-matching distractor antecedents (She remembered that the surgeon had pricked herself; Sturt, 2003). These structures are theoretically interesting because they have been found to affect regression and rereading behavior in eye tracking, and are thus promising candidates for an investigation of conscious rereading. We present all four sentence types to the same participants in one experiment.

If the selective reanalysis hypothesis holds, and if selective reanalysis is a deliberate, conscious reading strategy, the BSPR paradigm should reveal revisionary rereading of critical sentence regions in NP/Z and RRC garden-path sentences that results in positive acceptability judgments. Discourse inconsistencies (barks … kitten), on the other hand, should show confirmatory rereading with no effect on judgments, as there is no alternative structure or interpretation that can be derived. If participants miss the inconsistency on the first pass, judgments of discourse inconsistencies may even be negatively affected by rereading, as participants may become more certain that the discourse is indeed malformed. Finally, reflexive sentences may show both confirmatory and revisionary rereading, as we explain below.

To analyze first-pass reading and rereading in BSPR, one can compute reading measures that are familiar from the eye-tracking literature, such as first-pass reading times and rereading times. In addition to reporting such region-based measures, we also conduct a scanpath analysis. Scanpaths capture the trajectory of reading – usually in the form of eye fixations, but in our case in the form of key presses and reading times – across an entire trial. Different methods for comparing scanpaths between participants and conditions have been developed (see Anderson et al., 2015 for a review). We use the Scasim algorithm developed by Titus von der Malsburg,⁴ as it has previously been applied to garden-path sentences in eye-tracking and BSPR (Christianson et al., in prep.; von der Malsburg and Vasishth, 2011, 2013; Paape and Vasishth, 2022).

To ensure that our results replicate, we run two identical web-based BSPR studies with 100 participants each. We also run a standard SPR experiment in addition to the two BSPR experiments, in order to investigate the effects of the manipulations when rereading is not possible.

We now present the experimental setup and discuss each of the four manipulations in turn.

2. Materials

In order to increase the naturalness of reading and reduce participant fatigue, we use region-by-region as opposed to word-by-word SPR/BSPR. In all examples, the · symbol marks the boundaries between presentation chunks, that is, a key press is required at each · to reveal the next word or group of words.

3. Previous findings and predictions

For ease of reference, Table 1 gives an overview of all experimental manipulations.

4. Participants

For each of the two identical BSPR experiments (BSPR1 and BSPR2) and the standard SPR experiment, 100 self-identified native speakers of English were recruited over Prolific (https://www.prolific.co; Palan and Schitter, 2018). All participants indicated that they were currently living in the US. They received £ 3.50 for their participation.⁶

5. Procedure

The three experiments were run on the Ibex farm (https://spellout.net/ibexfarm; Drummond, 2013). Within each experiment, every participant read 92 sentences in total (24 NP/Z + 24 RRC + 20 Discourse + 24 Reflexive). Sentences were rotated through the experimental conditions according to a Latin-square design, and presentation order was randomized. Due to the amount and varied nature of the experimental materials, no fillers were used. Participants gave informed consent after reading the instructions, then completed two practice trials, then proceeded to the main experiment.

The procedure was the same as in Paape and Vasishth (2022), apart from the nature of the end-of-trial task: Paape and Vasishth asked comprehension questions and rewarded accuracy and reading speed with bonus payouts, whereas we collected acceptability judgments and offered no additional rewards. We chose acceptability judgments as our dependent measure because we were interested in the connection between selective rereading in a given trial and the end-of-trial judgment.

The instructions for the judgments were as follows:

When judging the acceptability of the sentences, please take into account both meaning and grammar. Sentences should be rejected as unacceptable if they violate the rules of English grammar (“Peter goed swimming yesterday”) and/or if their meaning is contradictory or implausible (“Mary ate a table for dinner yesterday”). Possible stylistic issues should be disregarded.

After each sentence, participants were presented with the question “Was the sentence acceptable?”. The possible responses were “Yes”, “No”, and “I don’t know”.

Because we were interested in the subjective naturalness of the BSPR paradigm compared to standard SPR, participants were presented with a debriefing screen after all sentences had been read and judged. Both in the BSPR studies and in the SPR study, participants were asked “Did reading feel natural in this experiment?” and “Were the sentences easy to understand?”. Only in the BSPR experiments, they were also asked “Was rereading helpful in this experiment?”. For each question, participants had four answer options: “Yes, very”, “Mostly yes”, “Mostly not”, and “Not at all”. Participants were also given the opportunity to comment on the experiment using a text box.

6. Data analysis

Data from seven participants (three in BSPR1, four in BSPR2) were removed because the participants either skipped a majority of sentences by using the CTRL key or indicated during debriefing that they had done an experiment with similar sentences before. From the remaining data, we removed all trials in which any single-region reading time was longer than 5000 ms or in which any region had not been read at all. This resulted in a loss of 7% of the data in both BSPR experiments and 6% of the data in the SPR experiment.

For every trial, we coded whether the acceptability judgment was positive or negative and, for BSPR only, whether the trial contained at least one regression. For each experiment, we fitted linear mixed-effects models (LMMs) with random slopes by participants and by items using the brms package for Bayesian inference (Bürkner, 2017, 2018) in R (R Core Team, 2020). Four chains with 4000 iterations each were run for all models. Acceptability judgments were fitted with a Bernoulli likelihood, using the factor condition as a predictor. The factor was sum-coded as follows:

• NP/Z ambiguity: Comma –1, No-comma +1

• RRC ambiguity: Animate NP –1, Inanimate NP +1

• Discourse inconsistency: Consistent continuation –1, Inconsistent continuation +1

• Reflexives: Mismatching distractor –1, Matching distractor +1

For the BSPR experiments, the overall proportion of regressions at the trial level (0 = no regression, 1 = at least one regression) was analyzed using a Bernoulli likelihood. In addition to the fixed effect of condition, the model contained the acceptability judgment at the end of the trial as a sum-coded predictor (unacceptable –1, acceptable +1), as well as the interaction between condition and judgment.

Our region-wise analysis of BSPR reading measures is focused on first-pass parsing and selective rereading, and closely follows that of Paape and Vasishth (2022). We selected a relatively small number of reading measures, given that each additional measure analyzed increases the risk of false-positive findings (von der Malsburg and Angele, 2017). We chose right-bounded reading time (RBRT), regressive rereading time (RegRRT), and the proportion of first-pass regressions from a region as our measures of interest. Right-bounded reading time is the sum of all reading times on the region before moving on to the next region; it is composed of first-pass reading time and progressive rereading time. We define regressive rereading time as the sum of all non-zero reading times on the region after having moved on to the next region, including both regressive and progressive revisits: After having read region n + 1, participants may either return to region n from the right, or jump back to the beginning of the sentence and return to region n from the left; both types of revisits count towards regressive rereading time for region n. Due to the exclusion of regions with RegRRT = 0 from the analysis, the measure is deconfounded from rereading probability (Leinenger et al., 2017; Vasishth and Drenhaus, 2011), and the data can be analyzed without violating distributional assumptions. Informative differences in rereading probability between conditions are captured by the scanpath analysis reported below.

The RBRT and RegRRT measures were analyzed using LMMs with a Lognormal likelihood. Following Paape and Vasishth (2022), we assume that data points below 150 ms in BSPR are equivalent to skipping a region. Removing these data points resulted in a loss of 2%/<1% of the data for RBRT and 84%/85% of the data for RegRRT, counting removed data points with RegRRT = 0. For both reading measures, data points above 5000 ms were treated as outliers and were also removed from the analyses, resulting in the loss of less than 1% of the data in both experiments.

For NP/Z, RRC and discourse inconsistency sentences, we defined a window of analysis that contained the critical region as well as the two regions to the left and one region to the right. The critical region in NP/Z sentences is the disambiguating verb, that is, wondered in (3). For RRC sentences, the critical region is the ambiguous participle, that is, sent in (4). For discourse inconsistency sentences, the critical region is the region containing the manipulated NP, that is the most alert puppy/kitten in (5).⁷ Finally, for reflexive sentences, the critical region is the region containing the reflexive, that is, had pricked herself in (6).

For all sentence types except RRC sentences, the sentence ended with the second region after the critical region in most (but not all) items. When the second-next region from the critical region was not the sentence-final region, the final region was analyzed instead. For reflexive sentences, we did not analyze reading times at the complementizer that, but instead included the pronoun She/He in the window of analysis, as readers could plausibly reread this region after encountering the matching or mismatching reflexive.

Rather than using a binary criterion to distinguish “present” from “absent” effects in the reading measures, we focus on quantifying uncertainty by presenting the estimates from our analyses with their accompanying uncertainty intervals (Vasishth and Gelman, 2021). When interpreting the results, the reader should consider the width of the credible intervals, and whether the effect is consistent across the experiments. The full range of results is given in the online supplement at https://osf.io/54bda.

7. Results – Debriefing

We recoded the debriefing responses numerically (“Yes, very” = 4, “Mostly yes” = 3, “Mostly not” = 2, “Not at all” = 1) and computed the mean ratings for each question for each method (BSPR vs SPR). BSPR was judged to be somewhat more natural than SPR (2.58 vs 2.34), and sentences were also judged to be somewhat more comprehensible in BSPR than in SPR (2.45 vs 2.24). A Bayes factor analysis shows anecdotal evidence of BSPR being considered more natural than standard SPR (BF = 2.23), as well as moderate evidence of BSPR being easier to comprehend than standard SPR (BF = 8.66). Rereading in BSPR was judged to be highly helpful (mean 3.38).

8. Results – Acceptability judgments and regressions

Table 2 shows the proportion of positive acceptability judgments, as well as trial-level regression probabilities (BSPR only) by experiment, sentence type and condition.

Figure 1a shows LMM estimates for the acceptability judgments. Acceptability judgments were reliably affected by the manipulations across the three experiments: NP/Z sentences without commas were accepted far less often than NP/Z sentences with commas. Sentences with inconsistent discourse continuations were accepted far less often than sentences with consistent discourse continuations. RRC sentences with inanimate NPs were accepted more often than RRC sentences with animate NPs, though the difference was much smaller in magnitude compared to NP/Z and discourse inconsistency sentences. Finally, reflexive sentences with matching distractors were accepted less often than reflexive sentences with mismatching distractors, but the difference between conditions was relatively small.

Figure 1

Effect estimates from the LMM analyses of acceptability judgments and regression probabilities.

Figure 1b shows LMM estimates for regression probabilities at the trial level. Across all sentence types, BSPR regressions occurred less often in trials with positive acceptability judgments compared to trials with negative acceptability judgments. NP/Z sentences without commas showed more regressions than NP/Z sentences with commas, while RRC sentences with inanimate NPs showed fewer regressions than RRC sentences with animate NPs. In BSPR1, NP/Z sentences showed an interaction, such that trials with positive judgments only showed fewer regressions in the comma condition, but there was no indication of such an effect in BSPR2. Across both experiments, discourse inconsistency sentences and reflexive sentences showed no indication that the manipulations affected regression probabilities at the trial level, despite a numerical tendency in discourse inconsistency sentences towards inconsistent continuations leading to more regressions.

9. Results – Reading measures

9.1 NP/Z sentences

Reading time measures for NP/Z sentences by experiment and sentence region are shown in Figure 2a (BSPR only). Figure 2b shows the estimates of the region-wise LMM analyses. Estimates from the standard SPR experiment are shown next to the RBRT results from the BSPR experiments. For a plot showing reading time measures by region for the SPR experiment, please refer to the online supplementary materials.

Figure 2

Reading measures and estimates from the LMM analyses for NP/Z sentences.

Despite some differences in the distribution of effects, the results converge across experiments: Readers slowed down when they encountered a comma after the ambiguous verb (trained(,)), that is, when the clause boundary was explicitly marked. This slowdown is presumably the result of being forced to adopt the dispreferred intransitive reading of the verb, or may simply be a low-level effect of processing punctuation (Warren et al., 2009). At the disambiguating main clause verb (wondered), a garden-path effect was visible in right-bounded BSPR reading times in the no-comma compared to the comma conditions, which in BSPR2 continued into the following spillover region. The garden-path effect was also visible in first-pass regressions at the point of disambiguation (wondered) and the two following regions. Regressive rereading times were elevated in the no-comma conditions in the NP region (the striker) and the disambiguating verb region. As Figure 2b shows, right-bounded reading times in BSPR closely align with reading times in standard SPR.

9.2 RRC sentences

Reading time measures for RRC sentences by experiment and sentence region are shown in Figure 3a (BSPR only). Figure 3b shows the estimates of the region-wise LMM analyses. Results from the standard SPR experiment are shown next to the RBRT results from BSPR.

Figure 3

Reading measures and estimates from the LMM analyses for RRC sentences.

The results for RRC sentences also largely converge across the two BSPR experiments: At the by-phrase and in the following region, participants read more slowly in sentences with animate NPs compared to sentences with inanimate NPs, consistent with a stronger garden-path effect. Right-bounded reading times in BSPR were again closely aligned with reading times in standard SPR.

In BSPR2 and SPR, there was some indication that readers slowed down at the verb (sent) when the preceding NP was inanimate, which contradicts the preferred active interpretation. Mainly in BSPR2, there was also some indication that stronger garden-pathing at the by-phrase in the animate-NP condition led to more regressions. A numerical difference between the experiments was visible across the entire window of analysis in regressive rereading times: In BSPR2, all theoretically relevant regions tended to be reread more in the animate-NP condition, while no such pattern was visible in BSPR1. Speculatively, this mismatch between the experiments may be due to different preferred reading strategies in the two samples of participants: As noted by Paape and Vasishth (2022) and Christianson et al. (in prep.), rereading strategies in response to garden-pathing may differ widely between individuals. Our scanpath analyses reported below shed additional light on the range of these differences. Alternatively, the differences between experiments may simply be due to noise in the rereading data, given that there were much fewer data points compared to the first-pass reading data.

9.3 Discourse inconsistency sentences

Reading time measures for discourse inconsistency sentences by experiment and sentence region are shown in Figure 4a (BSPR only). Figure 4b shows the estimates of the region-wise LMM analyses. Results from the standard SPR experiment are shown next to the RBRT results from BSPR.

Figure 4

Reading measures and estimates from the LMM analyses for discourse inconsistency sentences.

As for the garden-path sentences, the results for discourse inconsistency sentences mostly align across experiments: Encountering an inconsistent discourse continuation as opposed to a consistent discourse continuation led to longer right-bounded reading times in the manipulated region and the following regions, as well as more regressions. Right-bounded reading times in BSPR aligned with reading times in standard SPR.

One pattern that was very different across the two BSPR experiments was the effect of the manipulation on regressive rereading times at the critical region: BSPR1 showed a clear effect that is in line with selective rereading of inconsistent continuations, matching the findings of Connor et al. (2015), Zargar et al. (2020), and Hessel et al. (2021) in children. By contrast, BSPR2 showed no indication of such an effect. As for RRC sentences, this mismatch between the experiments may speculatively be due to different preferred reading strategies in the two participant samples or due to noise in the data.

9.4 Reflexive sentences

Reading time measures for reflexive sentences by experiment and sentence region are shown in Figure 5a (BSPR only). Figure 5b shows the estimates of the region-wise LMM analyses. Results from the standard SPR experiment are shown next to the RBRT results from BSPR.

Figure 5

Reading measures and estimates from the LMM analyses for reflexive sentences.

The results for reflexive sentences can be summarized quickly: There was no indication that any of the online reading measures in the theoretically interesting regions were affected by the manipulation of distractor match/mismatch. The numerical patterns observed in BSPR1 that could be of potential theoretical interest, that is, the patterns in right-bounded and regressive reading times of the critical reflexive region, were reversed in BSPR2. The only region that showed a consistent effect across the BSPR experiments was the final region of the sentence: The distractor-match conditions were more difficult to process at this point than the distractor-mismatch conditions. This effect is likely due to end-of-sentence wrap-up, which “attempt[s] to handle any inconsistencies that could not be resolved during the sentence” (Just and Carpenter, 1980, p. 345). Given that there was no indication of such an effect in standard SPR, it may be driven by progressive rereading.

10. Discussion

The results for NP/Z sentences indicate a very robust garden-path effect. Crucially, the garden-path effect was visible in first-pass regressions from the disambiguating region, as well as in regressive rereading times for the disambiguating region and the ambiguous NP. This is in line with the selective reanalysis hypothesis of Frazier and Rayner (1982), as well as with the proposal of Paape and Vasishth (2022) that selective reanalysis is conscious.

However, the large proportion of rejections in the offline judgments casts doubt on the selective reanalysis hypothesis: Assuming that selective reanalysis is usually successful and leads to positive acceptability judgments, one would have expected more than 40% positive judgments for NP/Z sentences, and rereading should have correlated with higher acceptability. The relatively low proportion of positive judgments suggests that readers engaged in selective rereading for confirmatory rather than for revisionary purposes. That is, they reread earlier regions mainly to check their syntactic analysis and/or their interpretation but did not necessarily revise it. We will return to this point after presenting the scanpath results.

Compared to NP/Z sentences, the differences between conditions were smaller for RRC sentences, and regressive rereading was not consistently affected by the animacy manipulation across the two BSPR experiments. Based on the end-of-trial judgment results, it appears that the NP/Z garden path is much stronger than the RRC garden path, which would be in line with previous findings (Van Schijndel and Linzen, 2021). NP/Z sentences without commas were rejected 60% of the time while RRC sentences with animate NPs were rejected only 20% of the time. It is thus possible that conscious, selective rereading is linked to the strength of the garden path as reflected in offline judgments: Garden paths that lead to more rejections may also show more selective rereading in general.

Discourse inconsistency sentences showed an interesting mismatch between online and offline measures compared to NP/Z sentences: The two manipulations had similar effects on the end-of-trial acceptability judgments, but the effect on reading times was much smaller for discourse inconsistency sentences than for NP/Z sentences, and there was no consistent evidence for selective rereading. Speculatively, this could be because participants sometimes outright fail to parse NP/Z sentences, while discourse inconsistencies can be parsed but are rejected as implausible. Compared to NP/Z sentences, readers need to consult their situation model in order to correctly accept or reject discourse inconsistency sentences (Hessel et al., 2021; Zargar et al., 2020; Connor et al., 2015; see van Moort et al., 2018 for a review), which may lead to more subtle effects compared to perceived ungrammaticality.

Finally, the results for reflexive sentences are compatible with the assumption that similarity-based interference affects end-of-sentence wrap-up, but does not cause conscious confirmatory or revisionary rereading during online processing. Given the mixed results in the literature for this construction (e.g., Dillon et al., 2013; Jäger et al., 2017; Parker and Phillips, 2017), it is not surprising that reflexives showed the weakest effects out of the four constructions tested.

11. Scanpath analysis of BSPR data

Scanpaths cover the trajectory of reading across the entire trial, and thus offer a more holistic view of the reading process than region-based measures. Scanpath analyses can uncover reading patterns that traditional region-based analyses may miss, including selective rereading (von der Malsburg and Vasishth, 2011; 2013; Christianson et al., in prep.; Paape and Vasishth, 2022).

We analyzed scanpaths for each sentence type separately by applying the Scasim algorithm for eye-tracking data (https://github.com/tmalsburg/scanpath; von der Malsburg and Vasishth, 2011) to the combined data from both BSPR experiments. Given that the total number of regions differed between individual sentences, we selected a subset of theoretically interesting regions for the scanpath analysis (see Figures 6 and 7 below) and dropped all visits to other regions from the data. Furthermore, for each trial, we considered only the part of the overall scanpath following the first visit to the critical region. In order to capture whole-sentence rereading, which is often observed in garden-path sentences (von der Malsburg and Vasishth, 2011, 2013; Paape and Vasishth, 2022), we included the first and last regions of the sentence in the analysis across all constructions. For the discourse inconsistency stimuli, we also added the inconsistent word in the first sentence (barks … kitten) to the analysis, as readers may regress to this word in order to check their interpretation.

Figure 6

Example scanpaths for NP/Z and RRC sentences.

Figure 7

Example scanpaths for discourse inconsistency and reflexive sentences.

Following Paape and Vasishth (2022), we first simulated horizontal eye movements based on the BSPR data by using region IDs as x-coordinates while keeping the y-coordinate constant, and computed scanpath similarities with Scasim. We then applied nonmetric multidimensional scaling (Kruskal, 1964a, b; Oksanen et al., 2020) to map the similarities into a 2-dimensional space, and finally used model-based clustering (Scrucca et al., 2016) to identify clusters of similar scanpaths. Please refer to Paape and Vasishth (2022) for further details regarding the procedure.

12. General discussion

The aim of this study was to investigate conscious rereading across different sentence types with the bidirectional self-paced reading (BSPR) paradigm. We were especially interested in distinguishing confirmatory rereading, which serves to increase the reader’s certainty in their existing interpretation, from revisionary rereading, which serves to change the existing interpretation. We tested four constructions: NP/Z garden-path sentences, RRC garden-path sentences, sentences with consistent or inconsistent discourse continuations, and sentences with reflexive anaphors that contain a structurally unavailable distractor antecedent.

All our manipulations were effective in the sense that they reliably affected end-of-sentence acceptability judgments. In addition, right-bounded reading times ín both of our BSPR experiments showed effects that are consistent with the existing eye-tracking literature: Ambiguous NP/Z sentences showed a strong garden-path effect at the point of disambiguation compared to unambiguous control sentences. RRC garden-path sentences were read more slowly at the point of disambiguation when the initial misanalysis was semantically plausible rather than implausible. Sentences with inconsistent discourse continuations showed a slowdown over several regions compared to sentences with consistent continuations. In reflexive sentences, which show conflicting results in the eye-tracking literature, the only consistent finding was that readers spent more time in the sentence-final region when the sentence contained a feature-matching distractor as opposed to a feature-mismatching distractor.

Despite evidence of processing difficulty across all sentence types, we found consistent evidence for selective rereading only in NP/Z garden-path sentences. The pattern of regressive rereading times in these sentences is compatible with the prediction of Frazier and Rayner (1982) that readers should focus mainly on the ambiguous portion of the string when first-pass parsing fails and syntactic reanalysis is triggered. The scanpath analysis confirmed that there were trials in which readers regressed from the point of disambiguation to the ambiguous region but no further, and then returned to progressive reading. Together, these findings provide the strongest direct evidence for selective rereading in garden-path sentences to date that we are aware of. The original results of Frazier and Rayner (1982) were descriptive in nature, and the observed regressions may have been due to readers trying to “buy time” as opposed to selectively targeting a specific sentence region (Mitchell et al., 2008). Mitchell et al. (2008) found some evidence of targeted rereading in NP/Z sentences, but the associated scanpaths were highly varied.⁹ Similarly, Meseguer et al. (2002) reported evidence of targeted rereading in Spanish garden-path sentences, but von der Malsburg and Vasishth (2011) could not identify a unique, selective scanpath pattern in Meseguer et al.’s data.

However, our results as a whole do not support the selective reanalysis hypothesis: There was no indication in the data that selective rereading helped readers find a grammatical analysis of the sentence, as evidenced by the large proportion of negative acceptability judgments following selective rereading. Thus, if a selective reanalysis mechanism exists, it would have to have a very high failure rate, which appears inconsistent with the claim by Frazier and Rayner (1982) that it is the parser’s default mechanism for dealing with temporary ambiguity. As noted by Fodor and Inoue (1994), the selective reanalysis hypothesis focuses on the diagnosis portion of garden-path recovery: The main challenge is to locate the source of the processing difficulty, and once the parser has succeeded, recovery is assumed to be straightforward. Thus, the presence of selective rereading in a given trial should indicate that diagnosis, and therefore recovery, was successful, contrary to what our data show.

Our result is in line with the findings of Christianson et al. (2017, in prep.), who found no evidence in their eye-tracking experiments that selective rereading is a frequent strategy, or that it leads to more successful comprehension of NP/Z sentences. The scanpath pattern associated with selective rereading was also very rare in our BSPR data: 4% of trials showed the selective strategy, and only about 30% of participants ever made use of it. The largest group of scanpaths that included rereading, encompassing the “varied rereading” and “reread from beginning” clusters, most frequently showed some variant of rereading the entire sentence.

Considering the absence of a positive effect of selective rereading on sentence acceptability, our results are more in line with a confirmatory view of selective rereading than with a revisionary view (Christianson et al., 2017, in prep.; Levy, 2008; Levy et al., 2009; Bicknell and Levy, 2011). Conscious, selective rereading at the sentence level may be limited to the NP/Z garden-path structure and possibly other extremely difficult structures. We found no consistent indication of selective rereading in RRC garden-path sentences, which are milder than NP/Z garden-path sentences (Van Schijndel and Linzen, 2021). The BSPR experiments of Paape and Vasishth (2022) showed no indication of selective rereading in German garden-path sentences, which are presumably also milder than NP/Z sentences. By contrast, studies that did find evidence of selective rereading are mostly limited to the NP/Z construction (Frazier and Rayner, 1982; Mitchell et al., 2008; Schotter et al., 2014), with the exception of Meseguer et al. (2002), who investigated adverbial clauses with ambiguous attachment in Spanish. This may suggest that there are two classes of garden paths: those that can be covertly or “automatically” reanalyzed and those that cannot. The underlying reason for such a distinction could be that the parser is missing the necessary reanalysis mechanisms for some structures (e.g., Marcus, 1980; Pritchett, 1992; Lewis, 1998) or that some structures involve early “pruning” of the correct alternative analysis, which thus cannot be recovered without conscious effort (see discussion in Meng and Bader, 2000). The NP/Z construction would appear to belong to this class of structures.

Another interesting property of the NP/Z structure became apparent from participants’ comments during debriefing. Ten participants commented that commas had been missing from some of the sentences, presumably referring to NP/Z sentences. Comma-less NP/Z sentences in combination with acceptability judgments have been used in a number of psycholinguistic studies (Van Dyke and Lewis, 2003; Tabor and Hutchins, 2004; Ferreira and Henderson, 1991; McElree, 1993). However, a check of several online grammar resources (Grammarly blog, 2021; Grammar.com, 2021; Purdue OWL, 2021) reveals that the use of a comma after an “introductory clause” (While the team trained, the striker wondered …) is considered mandatory, though exceptions may be made for “short” introductory phrases. In our BSPR sample, 13 participants (out of 193) always judged comma-less NP/Z sentences as unacceptable. Descriptively, these participants were more likely to use the selective rereading strategy in NP/Z sentences than the rest of the sample (10% versus 4% of trials). It is thus possible that selective rereading was partly due to participants checking whether a comma was present, and rejecting the sentence as ungrammatical when it was missing (see also Staub, 2007). However, even if all selective regressions in our data were triggered by the missing comma, our conclusion would not change: Conscious rereading appears to be due to the need to verify previously-read information, rather than by the need to reanalyze the syntactic structure of the sentence.

In addition, participants may have penalized sentences that were difficult to process and required rereading (Hofmeister et al., 2013), even though our instructions were to judge the sentences based on their grammaticality and meaning. Such a rereading penalty may partly explain the negative effect of rereading on acceptability that we observed across all sentence types. However, this account would still not favor the selective reanalysis hypothesis: If selective rereading is the parser’s default reanalysis mechanism, as claimed by Frazier and Rayner (1982), because it is arguably more efficient than rereading the entire sentence, it would be surprising if selective reanalysis incurred so much additional processing difficulty that participants end up rejecting the sentence. Alternatively, it could be that the NP/Z structure in particular is so difficult to reanalyze that it leads to rejections even when participants ultimately succeed at reanalysis. However, it was precisely the NP/Z structure that Frazier and Rayner used to argue for selective reanalysis, which makes a purely effort-based explanation of the negative judgments less convincing.

It might also be argued that any rereading observed in our experiments is an artifact of the judgment task, and that participants may have regressed even less if we had used comprehension questions or no task at all. Having participants read sentences without any task would be the only way to investigate reading uncontaminated by task effects. The downside is that one would lose all information about the final processing outcome. The usual way to deal with this issue is to use comprehension questions, which, however, can also affect reading strategies (Swets et al., 2008). Furthermore, it is difficult to establish a direct link between participants’ answers to comprehension questions and their syntactic analysis of a given sentence, as the responses may be affected by additional inferential and memory processes (Huang and Ferreira, 2021; Bader and Meng, 2018; Meng and Bader, 2021; Qian et al., 2018). Because our goal was to investigate the reading process along with the final processing outcome, we needed to add an offline task, in the same way as previous studies on selective reanalysis (Schotter et al., 2014; Meseguer et al., 2002; Frazier and Rayner, 1982). The linking assumption behind the judgment task was that participants give a negative judgment if they are unable to find a grammatical analysis of the string (Ferreira and Henderson, 1991; Warner and Glass, 1987). Investigating the effect of different tasks such as comprehension questions, acceptability ratings etc. on rereading patterns seems like an important future direction for psycholinguistics, but is beyond the scope of the current project.

Regarding the role of conscious awareness in rereading, it is tempting to conclude that the likelihood of deliberate, confirmatory rereading at the sentence level depends on the subtlety of the linguistic manipulation as reflected in the offline measures: Trials in which readers decide to reread parts of the sentence may be a subset of the trials in which readers decide that a sentence is malformed and ultimately reject it. In line with this view, the low offline acceptability of comma-less NP/Z sentences coincided with partial evidence of selective rereading during online processing. By contrast, RRC sentences with animate NPs and reflexive sentences with matching distractors showed relatively high acceptability and no evidence of selective rereading. However, the apparent connection between negative judgments and selective rereading is broken by sentences with inconsistent discourse continuations: As for NP/Z sentences, acceptability was low for this construction, yet there was no evidence of selective rereading. A plausible explanation for this mismatch is that difficulties that arise at the level of world knowledge (“Kittens do not bark”) are distinct from difficulties at the level of the syntactic analysis: Selective rereading may be more likely as a response to conscious parsing failure, because participants want to make sure that there is no correct alternative parse. By contrast, when the discourse is inconsistent (barks … kitten), it makes little sense to explore alternative analyses, given that the sentence is already well-formed from a purely syntactic point of view. Furthermore, it is unlikely that an adult native speaker would misread meows in the first sentence as barks, or puppy in the second sentence as kitten. Confirmatory rereading is thus not needed, but the sentence is still confidently rejected.

This perspective is in line with the proposal that readers are sensitive to the relative probabilities of different types of linguistic errors in their environment, such as misspelled, missing or superfluous words (Ryskin et al., 2018, 2021; Gibson et al., 2013). In addition, readers may be sensitive to the types of processing errors that they could have plausibly made while reading the sentence. Parsing errors appear to be intuitively judged as likely, and are thus more likely to trigger conscious, confirmatory rereading. By contrast, gross errors in word identification may be judged as relatively unlikely, and are thus less likely to trigger confirmatory rereading, at least in adults. Future work could investigate this possibility by collecting data about individual participants’ perceived relative probabilities of different processing errors, and correlating these with the corresponding online processing profiles.

A final salient question about our results is whether they are in line with the existing eye-tracking literature. The BSPR paradigm differs from eye tracking in that reading is more deliberate and less automatic. However, despite the clear differences between the paradigms, it is still informative to see whether the methods yield converging evidence for one and the same phenomenon. We focus here on first-pass regressions from the disambiguating region in NP/Z garden-path sentences (While the team trained(,) the striker wondered …). Figure 8 shows the estimated effect from our combined BSPR data from both experiments (193 subjects) compared to estimates from five published eye-tracking studies that report regression proportions in NP/Z sentences.

Figure 8

Comparison of garden-path effects in first-pass regression probabilities between five published eye-tracking studies and the current BSPR study. Error bars show 95% confidence intervals.

As the figure shows, the estimates from the eye-tracking studies of Ferreira and Henderson (1993), Pickering and Traxler (1998), and Mitchell et al. (2008) all have much larger uncertainty intervals than our BSPR study; these large uncertainties are due to the smaller sample sizes used in these studies (24–40 participants, cf. our 193 participants). Some of the experiments differed from ours not only in terms of the experimental method, but also with regard to the end-of-trial task (grammaticality judgments versus comprehension questions), and/or with regard to where in the sentence the ambiguity appeared (early versus late). Nevertheless, our estimate of a 0–3% increase in regressions at the disambiguating region is consistent with the overall empirical picture: The BSPR uncertainty interval is entirely contained in the uncertainty intervals from the earlier eye-tracking studies.

Despite the observed overlap between our BSPR-based estimate and those from the eye-tracking studies, it should be noted that the eye-tracking studies have a much higher baseline of regressions in unambiguous (comma) condition. For instance, in their Experiment 2, Mitchell et al. (2008) observed a regression rate of 18% at the critical verb wondered even in the comma condition (While the team trained, the striker wondered …). By contrast, in our BSPR study, the mean regression rate at this position in the comma condition was 2%. Compared to eye tracking, BSPR eliminates several potential sources of regressions, such as overshoot correction (Rayner, 1998) or “time-out” regressions that allow higher-level cognitive processes to catch up to the forward movement of the eyes (Mitchell et al., 2008; Engelmann et al., 2013). Overshoot correction in particular may be the source of many regressions in eye tracking: For instance, Eskenazi and Folk (2017) report a regression rate of 17% following an infrequent three-letter word (… wore a green gem around her neck …), 89% of which were classified as corrective rather than comprehension-mediated. Given that corrective regressions are likely automatic, it is thus not surprising that BSPR, which only allows deliberate regressions, has a much lower baseline regression probability. In future work, we aim to design experiments that specifically probe for conscious deliberation in eye-tracking regressions, which may uncover shared mechanisms between deliberate regressions in BSPR and in eye tracking.

13. Conclusion

Using an experimental paradigm in which regressions during rereading are carried out consciously, our study showed that selective rereading occurs only as a consequence of a difficult syntactic misparse (garden path), and that selective rereading, when carried out consciously, is confirmatory: The reader revisits earlier material selectively to confirm their initial analysis or to confirm the perceived ungrammaticality of the sentence, rather than to revise an initial misanalysis.

Data availability statement

All experimental materials, data and analysis code are available at: https://doi.org/10.17605/OSF.IO/54BDA.

Funding information

This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Grant 428960187 – PI: Dario Paape.

Competing interests

The authors have no competing interests to declare.

Author contributions

Dario Paape: Funding Acquisition, Conceptualization, Methodology, Software, Investigation, Formal Analysis, Writing – Original Draft, Project Administration; Shravan Vasishth: Funding Acquisition, Conceptualization, Writing – Review & Editing, Supervision.