Glossa Psycholinguistics

2.2 Results

Statistical analyses were conducted using the R statistics software (R Core Team, 2020). Reading time and reaction time data were analyzed with linear mixed-effects models, accuracy data were analyzed with generalized mixed-effects models, using the lme4 R-package (Bates, Mächler, et al., 2015). Models included the experimental factors and their interactions as fixed effects. Random effect terms were determined following the model fitting procedure proposed in Bates, Kliegl, et al. (2015). All factors were effect coded (0.5 vs. –0.5) and thus compared factor levels to each other. The questions for the filler sentences were answered with a mean accuracy of 98.9%. All participants had an accuracy of at least 83% and therefore no one was excluded from the analysis.

2.2.1 Reading times

The reading times were prepared for analysis in several steps, closely following Hofmeister (2011). First, extremely low or high reading times (rt < 100 ms or rt > 2500 ms) were removed. Second, reading times were log-transformed to avoid skewing. Third, residual reading times were computed from the log-transformed raw reading times, taking both experimental and filler items and contexts and target sentences into account. To obtain residual reading times, linear mixed-effects models were fitted to the log-transformed reading times with participants as random factor and the following fixed predictor variables: word length, the log-transformed position of the trial in the experiment, and the position of the word within the sentence. The latter factor was coded as a categorical variable with three levels: a word could either appear in sentence-initial position, in sentence-final position, or sentence-medial position (that is, in every linear position except the first and the final one). For analysis purposes, the target sentences were divided into several regions as shown in (4). In addition, the last word of the context sentence preceding the target sentence was analyzed (region S1-final). For each region, we report mean residual logarithmic reading times per word.

1. (4)

1. Target sentence regions for reading time analysis
2. 

Figure 1 shows the time course of reading the final word of the context sentence and the different regions of the target sentence. To make it easier to visualize the results, Figure 2 shows the same mean reading times with a single plot for each region, including error bars. The results of the linear mixed-effects models for the individual regions are summarized in Table 2.

Figure 1: Mean reading times for the last word of the context sentence and the different regions of the target sentence in Experiment 1.

Figure 2: Mean reading times for the last word of the context sentence and the different regions of the target sentence in Experiment 1. Error bars show 95% confidence intervals.

Table 2: Mixed-effects models for residual log reading times in Experiment 1.

Region	Contrast	Estimate	SE	df	t value	p value
S1-final	Order	0.008	0.022	1325.2	0.37	n.s.
	Context	0.109	0.022	1324.7	4.93	< 0.01
	Order × Context	–0.007	0.044	1325.4	–0.16	n.s.
NP1	Order	0.004	0.012	1334.7	0.32	n.s.
	Context	0.062	0.012	1334.5	5.11	< 0.01
	Order × Context	–0.009	0.024	1334.8	–0.39	n.s.
V-fin	Order	0.069	0.013	1336.3	5.42	< 0.01
	Context	–0.020	0.013	1336.0	–1.58	n.s.
	Order × Context	–0.048	0.025	1336.4	–1.90	< 0.10
NP2	Order	0.031	0.011	1335.1	2.77	< 0.01
	Context	–0.007	0.011	1334.8	–0.67	n.s.
	Order × Context	–0.026	0.022	1335.5	–1.16	n.s.
V-fin&NP2	Order	0.049	0.009	1334.5	5.24	< 0.01
	Context	–0.014	0.009	1334.3	–1.52	n.s.
	Order × Context	–0.038	0.019	1334.8	–2.02	< 0.05
Midfield	Order	0.051	0.009	1329.2	5.57	< 0.01
	Context	–0.013	0.009	1328.9	–1.40	n.s.
	Order × Context	0.032	0.018	1329.1	1.74	< 0.1
V-end	Order	0.072	0.021	1282.6	3.52	< 0.01
	Context	–0.066	0.021	1281.9	–3.19	< 0.01
	Order × Context	0.013	0.041	1282.2	0.31	n.s.

For the final word of the context sentence, a significant effect of Context was found due to higher reading times in supportive compared to neutral context sentences. On the initial NP of the target sentence (region NP1), reading times were again higher following supportive than following neutral contexts. Neither the factor Order nor the interaction between Order and Context were significant for NP1.

Reading times for the finite verb (region V-fin) were higher for OS than for SO sentences, resulting in a significant effect of Order, whereas Context was not significant and the interaction between Order and Context was marginally significant. On the NP following the finite verb (region NP2), OS sentences were again read slower than SO sentences, resulting in a significant effect of Order, but neither Context nor the interaction of Context and Order were significant. When V-fin and NP2 are combined to a single region, which can be considered the spill-over region of the sentence-initial NP, Order and the interaction of Order and Context are significant. This interaction reflects the finding that context had no effect on SO sentences in the joint V-fin/NP2 region (–0.048 vs. –0.044; t = 0.89, n.s.), but on OS sentences, which were read faster in this region when preceded by a supportive context (–0.012 vs. 0.022; t = 3.53, p < 0.01).

In the region following the second NP, which contains all words after NP2 except for the clause-final verb (region Midfield), Order was significant, the interaction of Order and Context was marginally significant whereas Context was not. In this region, the interaction reflects the finding of a context effect for SO sentences, which were read faster in supportive contexts (–0.003 vs. 0.030; t = 2.52, p < 0.05), but no context effect for OS sentences (0.062 vs. 0.065; t = 0.28, n.s.). The sentence-final verb (region V-end) was read faster in supportive contexts than in neutral contexts, resulting in a main effect of Context. In addition, the final verb was read faster in SO sentences than in OS sentences, resulting in a main effect of Order. The interaction of Order and Context was not significant.

2.2.2 Comprehension questions

Figure 3 shows the accuracy of answering the comprehension questions as well as the mean reaction times for correct answers. The corresponding mixed-effects models are summarized in Tables 3 and 4. The accuracy data show significant main effects of Order and Question Type as well as a significant interaction between Order and Question Type. The factor Context was neither significant as a main factor nor involved in any significant interaction. As can be seen in Figure 3, accuracy was high for SO sentences followed by an agent question but low when followed by a patient question. Accuracy for OS sentences was also low, but still somewhat higher than for SO sentences followed by a patient question. Pairwise comparisons revealed the following ranking between the four combinations of order and question type: Accuracy was higher for SO order/agent questions than for OS order/agent questions (89% vs. 69%; z = 6.77, p < 0.01); OS order/agent questions resulted in a higher accuracy than OS order/patient questions, but the difference was not significant (69% vs. 62%; z = 0.77, p > 0.1); the accuracy for OS order/patient questions, finally, was higher than the accuracy for SO order/patient questions (62% vs. 52%; z = 2.96, p < 0.01).

Figure 3: Percentages of correct answers to the comprehension questions and reaction times for correctly answered questions in Experiment 1. Error bars show 95% confidence intervals.

Table 3: Mixed-effects model for question accuracy in Experiment 1.

Contrast	Estimate	SE	z value	p value
Formula: correct ∼ QuestionType * Order * Context + (1 + QuestionType || participant) + (1 | sentence)
Intercept	0.921	0.149	6.20
Order	0.485	0.134	3.61	< 0.001
Context	0.131	0.131	0.99	n.s.
QuestionType	1.192	0.277	4.31	< 0.001
Order × Context	0.376	0.263	1.43	n.s.
Order × QuestionType	1.933	0.269	7.18	< 0.001
Context × QuestionType	–0.003	0.263	–0.01	n.s.
Order × Context × QuestionType	0.230	0.526	0.44	n.s.

Table 4: Mixed-effects model for reaction times for comprehension questions in Experiment 1.

Contrast	Estimate	SE	df	t value	p value
Formula: log(RT question) ∼ QuestionType * Order * Context + (1 | participant) + (1 | sentence)
Intercept	8.085	0.036	50.7	227.18
Order	–0.053	0.020	836.6	–2.61	< .01
Context	–0.014	0.020	829.4	–0.71	n.s.
QuestionType	–0.185	0.049	16.6	–3.81	< .01
Order × Context	–0.031	0.040	835.0	–0.76	n.s.
Order × QuestionType	–0.172	0.041	839.8	–4.24	< .01
Context × QuestionType	0.086	0.040	829.5	2.16	< .05
Order × Context × QuestionType	0.098	0.081	835.5	1.22	n.s.

Reaction times to correctly answered questions were longer for patient questions than for agent questions (3810 ms vs. 3064 ms), resulting in a main effect of Question Type, as shown in Table 4. Furthermore, the effect of Order was significant, but this effect is qualified by a significant interaction between Order and Question Type. For agent questions, reaction times were significantly longer after OS sentences than after SO sentences (3283 ms vs. 2901 ms; t = 5.26, p < 0.01). For patient questions, reaction times were longer for SO sentences than for OS sentences, but this difference did not reach significance (3889 ms vs. 3744 ms; t = 1.08, p > 0.1). In addition, the interaction between Context and Question Type was significant. This reflects the finding of longer reaction times for agent questions in neutral than in supportive contexts whereas the reverse holds for patient questions. However, the difference between reaction times in neutral versus supportive contexts was not significant for agent questions (3136 ms vs. 2993 ms; t = 1.11, p > 0.1) and only marginally significant for patient questions (3879 ms vs. 3745 ms; t = 1.89, p = 0.059).

2.2.3 Relationship between reading times and question accuracy

According to the GE model, misinterpretation errors observed for noncanonical sentences are the result of applying heuristics like the agent-before-patient heuristic, which are fast but error prone. Analyzing sentences with noncanonical word order using algorithmic procedures takes more time but typically delivers the correct interpretation. As argued above, the GE model therefore predicts that reading times for noncanonical sentences should be longer when the following question was answered correctly than when the question was answered incorrectly. Sentences with canonical word order receive the correct interpretation whether analyzed by heuristics or algorithmic means. For them, reading times should not differ depending on whether the following question was answered correctly or not.

In order to test the predicted relationship between reading times and correctness, Figure 4 shows mean raw reading times per word broken down by word order of the target sentence and correctness of the following wh-question. To test for significant effects, a linear mixed-effects model was fitted to the data, with logarithmic raw reading times as dependent variable, Order and Correctness as fixed effects, and participants and items as random effects. In accordance with the analysis presented above, reading times were longer for OS than for SO sentences, resulting in a main effect of Order (β = 0.049; SE = 0.011; t = 4.60, p < 0.01). Sentences followed by a correct answer were read somewhat faster than sentences followed by an incorrect answer, resulting in a main effect of Correctness (β = 0.027; SE = 0.012; t = 2.34, p < 0.05). The interaction between Order and Correctness was not significant (β = –0.0037; SE = 0.022; t = –0.15, n.s.).

Figure 4: Mean reading time per word for trials with correctly and incorrectly answered questions in Experiment 1.

2.2.4 Post-hoc analysis: Acceptability and choice preference of SO and OS sentences in context

We constructed the neutral and the supportive contexts of Experiment 1 based on our understanding of the relevant literature on word order variation in German. However, the finding that supportive contexts only had a weak facilitative effect on the processing of noncanonical sentences raises the question of whether the force of the supportive context to license OS order was too weak. To address this issue, we ran a post-hoc norming study after the main experiment was completed. Two different tasks were used to assess the force of the context to license OS order: acceptability ratings and a forced-choice continuation study (see Bresnan & Ford, 2010; Rosenbach, 2005). The tests were run on IBEX farm (Drummond et al., 2016) with different sets of participants.

For the rating study, context and target sentence were presented together on a single screen. Participants were asked to rate how well the second sentence fits the first sentence on a scale from 1 (“Does not fit at all”) to 7 (“Fits perfectly”). 18 students of Goethe University Frankfurt received course credit for participation in the test. The 24 experimental items were mixed with 42 filler items.

The mean acceptability values for the materials of Experiment 1 are shown on the left side in Figure 5. Ordinal mixed-effects models were fitted to participants’ responses using the ordinal package (Christensen, 2019). Results show significantly higher acceptability ratings for SO compared to OS sentences (z = 5.852, p < 0.01), and significantly higher ratings for sentences in supportive contexts than for sentences in neutral contexts (z = 7.410, p < 0.01). The interaction was significant as well (z = 2.937, p < 0.01), reflecting the finding that the acceptability of SO sentences improved in supportive contexts to an even greater extent than the acceptability of OS sentences.

Figure 5: Results of the post-hoc tests run to assess the force of a supportive context to license OS order in Experiment 1: Mean acceptability ratings in the acceptability rating test (left) and percentages of how often the OS continuation was chosen in the forced-choice continuation test (right).

In the forced-choice continuation study, a context sentence was always displayed together with two versions of the target sentence: one with SO order and one with OS order. Participants had to select the version of the target sentence which they would prefer as continuation of the context sentence. The order of presenting the two versions of the target sentence was a second factor (Presentation: SO-first vs. OS-first). 28 students from Goethe University Frankfurt received course credit for participation. The 24 experimental items were mixed with 36 items serving as fillers.

The percentages of selecting the OS target sentence as preferred continuation are shown on the right side of Figure 5. Overall, the OS target sentence was selected by participants in 18% of all cases. A logistic mixed-effects model revealed that neither the context nor the order of the two alternative continuations on the display (factor Presentation) had any significant effect. The interaction was not significant either. In sum, the results suggest that the supportive context condition improves acceptability of OS sentences as intended. However, the acceptability of SO sentences increased to an even stronger degree, and the strong preference for an SO continuation persisted. How these findings relate to the reading time results is considered in the discussion section, to which we turn now.

2.3 Discussion

In line with previous studies, Experiment 1 observed increased online processing difficulty for noncanonical OS sentences. With the exception of the sentence-initial NP, reading times for OS sentences were higher than reading times for SO sentences at all positions. The OS disadvantage was particularly robust at the sentence-final verb. The context manipulation affected reading times in several regions. On the sentence-initial NP, sentences following a supportive context elicited higher reading times than sentences following a neutral context. This effect is surprising, given that a supportive context established the sentence-initial NP as given, and evidence from prior research points to a processing advantage for given NPs compared to new NPs (Kaiser & Trueswell, 2004; Koizumi & Imamura, 2017). However, a similar effect was already visible on the last word of the preceding context sentence, possibly reflecting higher sentence wrap-up efforts due to the need to set up new discourse referents (Just & Carpenter, 1980). We therefore consider the effect of context on the sentence-initial NP a spillover effect.

On the finite auxiliary verb and the following NP2, context had no effect on reading times for SO sentences, but for OS sentences reading times were faster in supportive contexts than in neutral contexts, although the interaction was significant only when both regions were analyzed together. In the region following NP2, the context effect was reversed, with SO sentences now benefiting from a supportive context whereas OS sentences did not. On the sentence-final verb, a supportive context decreased reading times for SO and OS sentences to similar degrees. The facilitative effect of supportive contexts for both SO and OS sentences mirrors the results from the post-hoc acceptability test, which revealed higher acceptability values in supportive contexts for both SO and OS sentences.

In contrast to reading times as a measure of online processing, context had no main effect on how often comprehension questions were answered correctly, nor was it involved in any interaction. Order and Question Type, in contrast, strongly affected performance on comprehension questions, showing a robust interaction. For SO sentences, accuracy was much higher following agent questions than following patient questions. For OS sentences, accuracy was generally low, with no significant difference between agent and patient questions. For reaction times to comprehension questions, a similar pattern emerged. Longer reaction times were elicited in response to agent questions following OS compared to SO sentences. On the other hand, no reliable difference in reaction times was observed for patient wh-questions, for which reaction times were equally high following SO and OS target sentences.

The analysis of reading times depending on correctness on comprehension questions revealed increased reading times for OS sentences regardless of whether the comprehension question was answered correctly or not. We also found an effect of correctness. Sentences that were answered correctly were read faster than sentences that were answered incorrectly. The interaction between order and correctness was not significant.

In sum, the results of Experiment 1 show a robust effect of word order on online processing. The context manipulation had some moderate effects on the reading times but did not affect the comprehension accuracy for either SO or OS sentences. To follow up on the results obtained in Experiment 1, Experiment 2 tests whether the online and offline result patterns change if discourse properties provide stronger cues in favor of OS order.

3.2 Results

One participant was excluded from the analysis because of only 63% accuracy for the questions following the filler sentences (range of remaining 59 participants: 80%–100%, mean = 97.6%).

3.2.1 Reading times

Logarithmic residual reading times were computed in the same way as for Experiment 1. Figure 7 and Figure 8 show the time course of reading the final word of the context sentence and the different regions of the target sentence. The results of the linear mixed-effects models for the individual regions are summarized in Table 6.

Figure 7: Mean reading times for the last word of the context sentence and the different regions of the target sentence in Experiment 2.

Figure 8: Mean reading times for the last word of the context sentence and the different regions of the target sentence in Experiment 2. Error bars show 95% confidence intervals.

Table 6: Mixed-effects models for residual log reading times in Experiment 2.

Region	Contrast	Estimate	SE	df	t value	p value
S1-final	Order	–0.008	0.018	1302.3	–0.43	n.s.
	Object	–0.005	0.018	1302.6	–0.27	n.s.
	Order × Object	–0.034	0.037	1302.3	–0.92	n.s.
NP1	Order	0.022	0.012	1318.7	1.93	< 0.10
	Object	–0.012	0.012	1318.9	–1.02	n.s.
	Order × Object	–0.053	0.023	1318.8	–2.32	< 0.05
V-fin	Order	0.043	0.013	1332.8	3.27	< 0.01
	Object	0.019	0.013	1332.9	1.47	n.s.
	Order × Object	0.007	0.026	1332.8	0.27	n.s.
NP2	Order	0.038	0.010	1327.9	3.83	< 0.01
	Object	0.006	0.010	1328.2	0.58	n.s.
	Order × Object	–0.007	0.020	1328.0	–0.35	n.s.
Midfield	Order	0.021	0.009	1317.0	2.45	< 0.05
	Object	–0.000	0.009	1315.4	–0.03	n.s.
	Order × Object	–0.028	0.017	1315.1	–1.64	n.s.
V-end	Order	0.096	0.021	1300.6	4.64	< 0.01
	Object	–0.001	0.021	1300.6	–0.04	n.s.
	Order × Object	–0.034	0.041	1300.7	–0.81	n.s.

For the final word of the context sentence, no significant effects were observed, which was expected given that the context was identical across conditions. On the sentence-initial NP (region NP1), the effect of Order was marginally significant, the effect of Object was not significant, but the interaction was significant. The interaction reflects the finding that for sentences with a definite object, reading times on NP1 were significantly longer for OS than for SO sentences (0.083 vs. 0.033; t = 3.01, p < 0.01), whereas no significant reading time difference depending on word order showed up for sentences with a demonstrative object (0.048 vs. 0.043; t = 0.28, n.s.). For all remaining regions, the factor Order showed a significant effect whereas neither the factor Object alone nor the interaction between Order and Object was significant.

3.2.2 Comprehension questions

Accuracy and reaction times for answering questions in Experiment 2 were analyzed in the same way as in Experiment 1. As revealed by Figure 9, the percentages of correct answers and the reaction times for correct answers show the same patterns as in Experiment 1. The generalized mixed-effects model for the accuracy results, summarized in Table 7, again revealed significant main effects of Order and Question Type as well as a significant interaction between them. Accuracy was higher for SO order/agent question than for OS order/agent question (91% vs. 78%; z = 4.880, p < 0.01). Accuracy for OS order/agent question in turn was higher than accuracy for OS order/patient question (78% vs. 65%; z = 3.891, p < 0.01). Finally, OS order/patient question resulted in a higher accuracy than SO order/patient question (65% vs. 53%; z = 3.645, p < 0.01).

Figure 9: Percentages of correct answers to the comprehension questions and reaction times for correctly answered questions in Experiment 2. Error bars show 95% confidence intervals.

Table 7: Mixed-effects model for question accuracy in Experiment 2.

Contrast	Estimate	SE	z value	p value
Formula: correct ∼ QuestionType * Order * Object + (1 | participant) + (1 | sentence)
Intercept	1.340	0.165	8.12
Order	–0.257	0.146	–1.75	< 0.1
Object	0.068	0.146	0.47	n.s.
QuestionType	–1.693	0.173	–9.79	< 0.001
Order × Object	0.475	0.293	1.62	n.s.
Order × QuestionType	1.759	0.295	5.96	< 0.001
Object × QuestionType	–0.339	0.293	–1.16	n.s.
Order × Object × QuestionType	–0.900	0.586	–1.54	n.s.

The factor Object was not significant, nor was any interaction involving Object. An inspection of Figure 9 shows that accuracy was almost identical for sentences with definite NP and sentences with demonstrative NP in three combinations of the factors Order and Question Type (difference < 2%), but in the condition OS order followed by an agent wh-question, accuracy was about 11% higher for sentences with demonstrative NP than for sentences with definite NP (84% versus 73%). If reliable, this should have resulted in a three-way interaction, but with p = 0.12 the three-way interaction failed to reach significance. Since getting a significant three-way interaction is difficult when it is carried by a single, relatively small difference, we also computed two-way analyses separately for SO and OS sentences. For SO sentences, there was only a significant main effect of Question Type (β = 2.748; SE = 0.257; z = 10.68, p < 0.01). For OS sentences, in contrast, there was a significant main effect of Question type (β = 0.763; SE = 0.202; z = 3.79, p < 0.01) and also a significant interaction between Object and Question Type (β = 0.749; SE = 0.361; z = 2.07, p < 0.05). We therefore tentatively conclude that answering agent questions following an OS sentence is eased when the fronted object is a demonstrative NP.

Reaction times to correct answers are also shown in Figure 9. Although a pattern similar to Experiment 1 is visible, the corresponding linear mixed-effects model for the logarithmic reaction times, summarized in Table 8, reveals only a main effect of Question Type. Agent questions were answered significantly faster than patient questions (3576 ms vs. 4431 ms).

Table 8: Mixed-effects model for reaction times for comprehension questions in Experiment 2.

Contrast	Estimate	SE	df	t value	p value
Formula: log(RT question) - QuestionType * Order * Object + (1 | participant) + (1 | sentence)
Intercept	8.192	0.035	65.1	232.64
Order	0.029	0.021	907.1	1.36	n.s.
Object	0.027	0.021	904.8	1.31	n.s.
QuestionType	0.161	0.033	25.0	4.95	< 0.001
Order × Object	0.001	0.043	905.4	0.02	n.s.
Order × QuestionType	–0.056	0.043	909.6	–1.31	n.s.
Object × QuestionType	0.050	0.043	905.8	1.17	n.s.
Order × Object × QuestionType	0.020	0.086	905.6	0.23	n.s.

3.2.3 Relationship between reading times and question accuracy

As for Experiment 1, we analyzed the relationship between reading times, taken as mean raw reading time per word, and the correctness of the answer selected for the wh-question following each target sentence. As shown in Figure 10, reading times were higher for OS than for SO sentences, in line with the reading time analysis presented above. Figure 10 also shows small differences between sentences followed by correctly and incorrectly answered questions. However, while the factor Order was significant (β = 0.047; SE = 0.010; t = 4.80, p < 0.01), neither the factor Correctness (β = 0.013; SE = 0.011; t = 1.18, n.s.) nor the interaction between Order and Correctness (β = 0.028; SE = 0.020; t = 1.41, n.s.) was significant.

Figure 10: Mean reading time per word for trials with correctly and incorrectly answered questions in Experiment 2.

3.3 Discussion

Consistent with Experiment 1, Experiment 2 found that OS sentences elicited longer reading times compared to SO sentences. For sentences with a definite object, this effect surfaced in all regions starting with the initial NP, and it was again particularly robust on the sentence-final verb. For sentences with a demonstrative object, the reading time penalty for OS order started at the finite verb. On the sentence-initial NP, however, reading times for OS sentences (initial NP = demonstrative object) and SO sentences (initial NP = subject) did not differ. The online data for the sentence-initial NP thus mirror prior corpus and experimental findings, including the pretest results of Experiment 2, that demonstrative objects are particularly favorable for using OS order. The online data for the remaining part of the sentences, on the other hand, show a disadvantage for OS sentences in comparison to SO sentences, independent of the particular determiner used for the object.

The accuracy results look very similar to those of Experiment 1. Experiment 2 again found a robust interaction between Order and Question Type. For SO sentences, accuracy was much higher for agent questions than for patient questions, whereas the accuracy for OS questions was in between. In addition, the object manipulation led to a slight improvement of accuracy for OS sentences followed by agent questions. As in Experiment 1, agent questions were responded to substantially faster than patient questions.

In sum, Experiment 2 provides further evidence that discourse-related properties licensing noncanonical order alleviate online processing difficulty for OS sentences, although restricted to the sentence-initial NP. Comprehension accuracy improved marginally at best for OS sentences when OS order was favored by discourse-related properties. Instead, accuracy was, as in Experiment 1, jointly determined by word order of the target sentence and question type.

4.2 Results

Data preparation and statistical analysis proceeded in the same way as in the preceding experiments. The questions for the filler sentences were answered with a mean accuracy of 94.4%. All participants had an accuracy of at least 70% and therefore no one was excluded from the analysis.

4.2.1 Reading times

Figure 11 presents the reading times in each region broken down by the single factor Order. The results of the mixed-effects models for each region are summarized in Table 10. For the final word and the initial NP of the target sentence, the factor Order did not have a significant effect. For the finite verb, a marginally significant effect of Order is observed, with OS sentences leading to longer reading times than SO sentences. The next two regions, NP2 and Midfield, show again no significant effect of Order, but for the clause-final verb, reading times were significantly higher for OS than for SO sentences.

Figure 11: Mean reading times for the different regions of the target sentence in Experiment 3. Error bars show 95% confidence intervals.

Table 10: Mixed-effects models for residual log reading times in Experiment 3.

Region	Contrast	Estimate	SE	df	t value	p value
S1-final	Order	–0.011	0.030	700.7	–0.36	n.s.
NP1	Order	–0.017	0.017	703.3	–1.01	n.s.
V-fin	Order	0.029	0.016	709.2	1.79	< 0.10
NP2	Order	0.012	0.014	708.4	0.87	n.s.
Midfield	Order	0.009	0.012	705.3	0.76	n.s.
V-end	Order	0.064	0.028	701.9	2.31	< 0.05

4.2.2 Comprehension questions

Figure 12 shows the percentages of comprehension questions that were answered correctly in Experiment 3 and the reaction times for correct answers. The corresponding mixed-effects models are summarized in Tables 11 and 12. Overall, questions were answered correctly with an accuracy of 83%. As shown in Figure 12, accuracy was somewhat higher for SO than for OS sentences (86% versus 79%), resulting in a significant effect of Order. The factor Question Type was not significant, nor the interaction between Order and Question Type. The mixed-effects model for the reaction times for correctly answered questions revealed significant effects of the two main factors and a significant interaction between them. While Question Type did not have a significant effect on SO sentences (2882 ms vs. 2839 ms; t = 0.33, p > 0.1), it affected OS sentences significantly, with agent questions needing about 500 ms longer for correct answers than patient questions (2801 ms vs. 3301 ms; t = 3.25, p < 0.01).

Figure 12: Percentages of correct answers to the comprehension questions and reaction times for correctly answered questions in Experiment 3. Error bars show 95% confidence intervals.

Table 11: Mixed-effects model for question accuracy in Experiment 3.

Contrast	Estimate	SE	z value	p value
Formula: correct - QuestionType * Order + (1 | participant) + (1 | sentence)
Intercept	2.027	0.278	7.28	n.s.
Order	–0.588	0.209	–2.78	< 0.01
QuestionType	0.057	0.209	0.27	n.s.
Order × QuestionType	0.339	0.416	0.81	n.s.

Table 12: Mixed-effects model for reaction times for comprehension questions in Experiment 3.

Contrast	Estimate	SE	df	t value	p value
Formula: log(RT question) ∼ Order * QuestionType + (1 | participant) + (1 | sentence)
Intercept	7.933	0.043	41.4	183.50
Order	0.057	0.027	563.8	2.13	< .05
QuestionType	–0.069	0.027	562.4	–2.58	< .05
Order × QuestionType	–0.114	0.053	561.2	–2.13	< .05

4.3 Discussion

Experiment 3 again found longer reading times for OS sentences compared to SO sentences. However, in contrast to the preceding experiments, the disadvantage for OS sentences was only significant on the finite verb and on the clause-final verb. On the sentence-initial NP1, reading times for SO sentences were slightly higher than reading times for OS sentences, but the difference was not significant. This is consistent with the absence of an effect of Order on NP1 in Experiment 2 for sentences with a demonstrative NP.

The main finding yielded by Experiment 3 is that presenting one-argument wh-questions instead of two-argument questions, as in Experiment 1 and 2, eliminated any effects of question type on the accuracy of answering comprehension questions, although an effect was still visible on the reaction times. Accuracy on comprehension questions was fairly high overall, both for agent and for patient questions. Note also that the accuracy values are close to the values obtained for SO sentences and active questions in the demonstrative object NP condition in Experiment 2.

5.1 Noncanonical sentences in context

In line with previous findings, our online results show that OS sentences are more difficult to process than SO sentences, and that this difficulty can be reduced by discourse properties licensing OS order. Facilitative effects showed up mainly at early sentence positions: on the finite verb and the following NP when the context was manipulated to license OS (Experiment 1), and on the sentence-initial NP when the referential form of the object NP favored OS order as well (Experiments 2 and 3). In later sentence regions, however, a disadvantage for OS sentences emerged regardless of context or object type, though less pronounced in Experiment 3. Thus, discourse properties licensing OS order temporarily alleviated, but did not eliminate, processing difficulty for OS sentences.

Our finding of facilitative context effects in early sentence regions is consistent with prior studies showing that context can exert effects while a noncanonical sentence is being processed (Kaiser & Trueswell, 2004) and in particular before the main verb is reached (Bornkessel et al., 2003; Koizumi & Imamura, 2017; Yano & Koizumi, 2018). Our results also confirm studies which demonstrated that effects of context on online processing are not restricted to cases of syntactic ambiguity, but affect unambiguous sentences as well (Grodner et al., 2005).

Results across studies differ with respect to whether the facilitative effect of context is temporary, with a disadvantage for noncanonical sentences emerging later in the sentence regardless of whether context supports OS order or not (as in Koizumi & Imamura, 2017, and the experiments reported here), or whether the facilitative effect of context persists until the end of the sentence (Kaiser & Trueswell, 2004). Note, however, that Kaiser and Trueswell (2004) examined Finnish, an SVO language, whereas Koizumi and Imamura (2017) and the experiments reported here tested SOV languages and used structures in which object and subject NP were processed before reaching the main verb.

In contrast to the online data, discourse properties affected comprehension accuracy only in a very limited way. Experiment 1 showed no effect of context on comprehension accuracy at all. In Experiment 2, accuracy was slightly higher for OS sentences with a demonstrative object than for OS sentences with a definite object, but only when the question asked for the agent of the sentence. In this case, accuracy reached a value of 84%, which is not much below the 90% reached by SO sentences followed by an agent question. Using the same contexts and demonstrative objects, but syntactically less complex questions, Experiment 3 revealed high accuracy values for both agent and patient questions, with only a small disadvantage for OS sentences in comparison to SO sentences. We therefore conclude that the low accuracy for comprehension questions observed in some conditions in Experiments 1 and 2 cannot be attributed to systematic misinterpretation, in line with conclusions drawn in Paolazzi et al. (2019), Bader and Meng (2018), and Cutter et al. (2022). We will advance a proposal concerning the source of the observed misinterpretation errors in the next section.

Regarding the online data, an account is needed why OS sentences induce increased processing cost even when OS order was as acceptable as SO order, as confirmed by the pretest results of Experiment 2. Current discussions of sentence complexity have revolved around the distinction between memory-based and expectation-based explanations of comprehension cost (see the overview in Levy, 2013). As discussed in Levy (2013), sentence comprehension is subject to expectation and memory effects, sometimes even when considering a single construction type (for German, see Levy & Keller, 2013, and Vasishth & Drenhaus, 2011). The question thus is what type of effect(s) we are seeing in our reading time data.

According to surprisal theory (Levy, 2008) – an influential expectation-based model – the processing cost associated with a word is higher the less expected the word is given the preceding context. As research on English subject and object relative clauses has shown, surprisal theory typically predicts a reading time penalty at the beginning of sentences with noncanonical word order whereas at later sentence positions either no difference or even an advantage for noncanonical sentences is predicted. Applied to German SO or OS main clauses, surprisal theory makes three predictions that are similar to the predictions for English relative clauses. First, sentences start much more often with a subject than an object. In her corpus study of written German, Hoberg (1981, p. 162) found that about 58% of all main clauses started with the subject, but only about 3.5% started with an object (the rest started with an adverbial or a predicate nominal). Reading times should therefore be higher for OS than for SO sentences on the first NP unless the preceding context makes the OS structure as likely as the SO structure. Second, the likelihood of seeing a subject is high after having seen an object, because almost all sentences with an object also have a subject. The likelihood of seeing an object after having seen a subject is not as high because subjects do not have to be accompanied by an object, as in intransitive sentences, for example. Reading times should therefore be higher for SO than for OS sentences on the second NP. Third, after both subject and object have been encountered, no further differences between SO and OS sentences are expected because the likelihood of what follows subject and object should be independent of the order between them.

The reading time data yielded by the three experiments reported in this paper are only in partial agreement with these three predictions. In accordance with surprisal theory, we found a smaller or no reading time penalty at the beginning of OS sentences when the context and the object’s determiner supported the use of OS sentences. On the second NP, there was either no difference or a penalty for OS sentences. For the rest of the sentences, there was also either no difference or a penalty for OS order, and in all three experiments, the disadvantage for OS sentences was particularly pronounced on the clause-final verb. In sum, while surprisal theory is compatible with the early effects of context that we found, the increased reading times for OS sentences at later positions cannot be explained in terms of surprisal.

Note that our results differ from the findings of Grillo et al. (2018) on the relationship between German active and passive clauses. In two selfpaced reading experiments, Grillo et al. found faster reading times for the second NP in passive clauses, that is, the NP within the by-phrase of a passive clause, than the second NP in active SO sentences, that is, the direct object. The authors explain this finding in terms of surprisal – the second NP is highly predictable in a passive clause after the parser has already processed the passive auxiliary and the preposition von (‘by’), whereas the direct object is not predictable. There are several possible non-exclusive reasons for the lack of a comparable effect in our experiments – slower reading times on the subject in OS sentences than on the object in SO sentences. First, although the subject is highly expected in an OS sentence, its position is variable; it can follow the finite verb immediately but it can also be separated from it by adverbials. Furthermore, passive clauses are less marked than OS clauses because they still start with the subject.

Memory-based explanations are the major alternative to expectation-based explanations. According to the Dependency Locality Theory (DLT) (Gibson, 2000), two memory components contribute to processing cost, storage cost and integration cost. Storage costs reflect open dependencies that are awaiting completion and must therefore simultaneously be stored in working memory. For SO and OS sentences as investigated in Experiments 1–3, storage costs differ at the initial NP and the finite verb depending on order. At these early positions, there is a storage cost of one for SO sentences because a main verb is still needed for sentence completion. OS sentences, in contrast, come with a storage cost of two because in addition to a main verb, a subject NP is needed in a complete sentence. When the second NP is processed, the subject is found. From the second NP onward, SO and OS sentences are therefore both associated with a storage cost of one until the end of the sentence is reached and the main verb leads to sentence completion.

Integration costs, the second processing cost component postulated by the DLT, measure the amount of processing needed to integrate each newly encountered input word into the structural representation built thus far. A main contributor to integration cost is the number of new discourse referents intervening between the first and the second element of a syntactic dependency. For the sentences under consideration, (8) shows the two dependencies relevant for the current discussion.²

The subject and the finite verb are connected by a dependency because the finite verb assigns nominative case to the subject and agrees with it in number and person. The dependency between object and main verb is necessary because the verb assigns case to its object.

Since nothing intervenes between subject and finite verb – both are adjacent to each other in SO and OS sentences – no order-dependent processing differences are predicted at early sentence positions. The dependency between object and main verb has to be computed at the end of the sentence when the main verb is encountered in the input. In an SO sentence, only adverbial material intervenes between object and main verb so that the integration depends on how many new discourse referents are introduced by the adverbial(s). In example (8), one new referent is introduced (Maske ‘mask’). In OS sentences, the same adverbial material intervenes between object and main verb, but the subject also intervenes, so integration costs are always higher by one in OS sentences in comparison to SO sentences, independent of the complexity of the following adverbial material. Integration cost at the sentence-final main verb is thus higher in OS than in SO sentences.

In sum, a memory-based explanation along the lines of the DLT of Gibson (2000) could explain several of the findings yielded by the present experiments. Storage costs are higher for OS sentences at the beginning of a sentence and equal for SO and OS sentences at later positions. Integration costs, on the other hand, are equal up to the clause-final verb where they are higher for OS sentences. However, certain findings remain unaccounted for under the DLT. First, the discourse-based attenuation of the reading time disadvantage for OS sentences is not captured by the memory-based DLT. Here, one could combine Surprisal Theory and DLT to also account for this finding, as has been done for other structures by, e.g., Levy and Keller (2013); Vasishth and Drenhaus (2011). The one remaining finding that is neither accounted for by the DLT nor by Surprisal Theory is the finding of prolonged reading times for OS sentences from the second NP onward up to the clause-final verb in Experiments 1 and 2 but not in Experiment 3, where the OS disadvantage showed up only on the clause-final verb. This could indicate that the complexity of the secondary task required after reading a sentence may also contribute to specific differences in reading time patterns. Although all three experiments reported here used the same task of answering wh-questions, the questions themselves were less complex in Experiment 3 than in Experiments 1 and 2. This raises the possibility that small and localized differences between SO and OS sentences get magnified when participants become aware that sentences are followed by questions that are sometimes difficult to answer. More research examining different task requirements is needed to clarify this issue.

5.2 The source of misinterpretation errors

Our experiments revealed a robust effect of question type on the accuracy of comprehension questions. For both SO and OS sentences – though for OS sentences to a lesser extent – the number of misinterpretation errors varied depending on the particular form of the question. For ease of reference, the accuracy results are summarized in (9).

In Experiments 1 and 2, accuracy was highest for SO sentences followed by an agent question and lowest for SO sentences followed by a patient question. Accuracy for OS sentences was in between, and again agent questions resulted in higher accuracy than patient questions. Experiment 3 used questions with a single argument only and found equally high accuracy for agent and patient questions. In a mixed-effects model with Question Type (agent vs. patient question), Order (SO vs. OS), and Question Complexity (two arguments, Experiments 1/2 vs. one argument, Experiment 3) as fixed effects and participants and sentences as random effects, all main effects and all interactions except the interaction of Order and Question Complexity were significant. Pairwise comparisons conducted to explore the crucial interaction between Question Type and Question Complexity revealed that for two-argument questions, agent questions resulted in significantly more correct answers than patient questions (81% vs. 58%; β = 1.110; SE = 0.171; z = 6.49, p < 0.01) whereas for one-argument questions, there was no significant difference between agent and patient questions (82% vs. 83%; β = –0.060; SE = 0.202; z = –0.30, n.s.).

The question now is what causes the pattern of misinterpretation errors. According to the GE model, misinterpretation errors result from applying heuristics that are fallible in case of noncanonical sentences. As discussed above, the relatively high accuracy values for OS sentences in some conditions are not easy to account for in the GE model. Furthermore, we did not find the relationship between correctness and reading times expected under the GE account. However, as pointed out by a reviewer, it cannot be ruled out that the predicted relationship between reading times and correctness is obscured by additional processes that intervene between reading the sentence and answering the comprehension question. For example, participants may read sentences with a fast pace but then pause and silently “re-read” the sentence before giving their answer, thereby preventing any simple relationship between reading times and question accuracy. We must leave it to future research whether an amendment of the GE model along these lines is viable.

Overall, our findings fit well with accounts that ascribe misinterpretation errors to fallible memory mechanisms – to degrading memory representations (Paolazzi et al., 2019, 2021) or to fallible retrieval operations (Bader & Meng, 2018; Meng & Bader, 2021). A detailed account of what these retrieval operations and memory representations are and why they often fail in the case of noncanonical sentences is still lacking, however. As a first step toward a comprehensive retrieval account, we will now specify the individual processing steps that lead from a wh-question to the retrieval of a phrase from the memory representation of a target sentence. We first note that accessing the answer directly by means of a simple cue-based memory retrieval would not account for the error pattern. Accessing the answer using semantic role as a retrieval cue is not compatible with finding many errors with patient questions in Experiments 1 and 2 but not in Experiment 3. Syntactic function cannot act as a simple retrieval cue either, because a wh-subject matches the target sentence’s subject when the question is in the active voice but its object when the question is in the passive voice. A retrieval cue “subject” would thus be underspecified.

In the case of two-argument questions, question and target sentence could be matched sequentially – first NP with first NP and second NP with second NP. A corresponding parallelism effect is clearly visible for SO sentences: accuracy was very high for agent questions (= SO order) and very low for patient questions (= OS order). Interestingly, however, OS sentences did not show a parallelism effect. For them, accuracy was low in general, and it was even somewhat lower for the syntactically parallel OS patient questions.

Thus, a more intricate way of matching questions to target sentences is needed. An important clue as to how such a matching algorithm could work is provided by the finding that accuracy was always best when the wh-phrase was a subject: (i) In Experiments 1 and 2, accuracy for SO sentences was much higher for agent questions (wh-phrase = subject) than for patient questions (wh-phrase = object). For OS sentences, accuracy was also higher when the question asked for the subject, although the difference was smaller. (ii) When the patient was queried using a subject wh-phrase, as in the passive questions of Experiment 3, accuracy was equally high as for agent questions. (iii) Throughout, reaction times were substantially higher for OS patient questions than for SO agent questions.

In order to capture the better performance with subject questions, we propose that the subject of the question – whether it is the wh-phrase or not – serves as starting point when matching the question to the target sentence. After the subject has been matched, further arguments are matched if necessary. The overall algorithm for retrieving the answer given a wh-question is given in (10).

A further finding, which is not yet accounted for by the question answering algorithm in (10), is that for both agent and patient questions with two arguments, accuracy was higher when the phrase of the target sentence corresponding to the wh-phrase appeared sentence-initially (SO with agent questions, OS with patient question). In order to capture this additional finding, we assume that accessing a phrase within working memory is easiest for phrases in sentence-initial position. Independent evidence for this assumption comes from research on anaphora resolution (e.g., Gernsbacher & Hargreaves, 1988; Gordon et al., 1993) and research on sentence interpretation, including Ferreira (2003, p. 175), who found that participants were more accurate naming the agent than the patient for active sentences (agent-initial), and more accurate naming the patient than the agent for passives (patient-initial).

To see how the proposed algorithm works, let us work through the different combinations of target sentences and question types, as illustrated in Table 13. Agent questions following an SO target sentence are the simplest case. Because the subject of the question and the subject of the target sentence both occur sentence-initially, locating the phrase corresponding to the question’s subject is straightforward. Since in agent questions the subject is the wh-phrase, Step 2 can be skipped. The final step – retrieving the correct answer – is easy too, because the phrase in the target sentence that corresponds to the wh-phrase in the question occurs in sentence-initial position.

Table 13: Illustration of the wh-question answering algorithm in (10).

Consider next OS sentences followed by an agent wh-question. The first step is more involved in this case because the subject now occurs sentence-finally in the target sentence. Step 2 can be skipped again. In order to get the correct answer at Step 3, the sentence-final subject must be retrieved from the target sentence. Given the evidence discussed above that sentence-initial constituents are more accessible than sentence-final ones, retrieving the subject from an OS target sentence is a more complex operations than retrieving the subject from an SO target sentence.

We now turn to patient questions, which elicit more errors on comprehension questions and higher reaction times. The two cases to consider are shown in the lower part of Table 13. Here, the subject occurs in final position, so locating the subject within the question in order to start the matching process is more complex than in the case of agent questions. Furthermore, since the subject is not the wh-phrase, the alignment established during the first step does not already identify the phrase needed for the answer. The wh-phrase is processed during the second step, when the objects are aligned. Only then does the information needed for retrieving the answer become available. We hypothesize that the elevated reaction times for patient questions compared to agent questions reflect the need to first match the subject before the wh-object can be used to identify the answer phrase. As shown in Table 13, for patient questions OS sentences are at an advantage because the phrase that needs to be retrieved occurs sentence-initially, and also because of parallelism between question and target sentence.

In contrast to two-argument questions, one-argument questions were answered with the same high accuracy independently of whether the question asked for the agent or the patient. This follows from the proposed question answering algorithm because the wh-phrase in one-argument questions is the subject in both agent and patient questions.

Before concluding, we should point out that our account of question answering does not imply that object questions are always difficult to process. We consider it likely that the difficulties seen with object questions in Experiments 1 and 2 are also due to having the indefinite pronoun jemand (‘someone’) as subject. Because the indefinite pronoun is semantically impoverished, it is not of great help when matching the subject of the question to the subject of the target sentence, as required in Step 1 of the proposed algorithm. With a lexical subject instead, this step is much easier to accomplish, as illustrated in (11). Here, we would expect that both agent and patient questions are answered with high accuracy. Crucially, the patient question should still need more time to be answered than the agent question.

In sum, we hypothesize that providing an answer to a wh-question proceeds in several steps that are fallible to varying degrees, depending on properties of the question and properties of the target sentence. We have specified the individual steps in a qualitative way and must leave a formal implementation allowing for quantitative predictions as work for future research.

5.3 Conclusion

In three experiments, we have investigated the comprehension of sentences with either canonical SO or noncanonical OS word order in context, using selfpaced reading as online and comprehension questions as offline measures. The reading time results show an overall disadvantage for OS sentences that is attenuated when the context or the object’s determiner favors OS order. The offline results revealed that the accuracy of answering a question depended strongly on the syntactic form of the question. To capture the pattern of misinterpretation errors, we have proposed that participants understood the sentences correctly most of the time, but sometimes retrieved the false answer due to difficulties with matching question and target sentence.