The ability to use grammatical rules to combine lexical items (i.e., words or signs) into phrases and sentences forms the core of the human capacity for language (Chomsky, 2017; Friederici et al., 2017; Lenneberg, 1969). Within a sentence, individual phrases are hierarchically grouped into constituents, essentially, groups of lexical items that behave as units (Everaert et al., 2015). For example, in the English sentence in (1), the lexical items [the new and interesting book] form a complex constituent, which can be focused (i.e., made more prominent by moving it to the front) in a so-called cleft construction, as in (2):

This does not apply to the non-unit [new and interesting book in], which is why the construction in (3) is considered ungrammatical (conventionally indicated by an asterisk):

The theoretical necessity of such groupings of lexical items into constituents for adequate descriptions of linguistic structures and their relations to each other was recognised early, and by now is firmly established (Everaert et al., 2015; Graffi, 2001; Harris, 1946; Nida, 1948; Pike, 1943; Wells, 1947).

Research over the past decades has established that the grammatical structure of sign languages also relies on the hierarchical grouping of lexical items into constituents (Cecchetto, 2017; Kocab et al., 2023; Mathur & Rathmann, 2014; Sandler & Lillo-Martin, 2001, 2008; Tang & Lau, 2012). For example, in the German Sign Language (DGS) sentence in (4), only the complete constituent [BOOK NEW INTERESTING] can be topicalized (accompanied by a non-manual marker in the form of raised eyebrows, glossed as ‘top’) by moving it to the front, as in (5).

This is not true for [BOOK NEW] as a sub-unit of the constituent which is embedded in the complex adjective phrase, so that extraction results in (6), which is considered ungrammatical.

The basic principles upon which language-specific syntactic structures are created are, therefore, remarkably similar across modalities (i.e., speech and sign), which is why some researchers treat them as abstract and largely universal (Chomsky, 1995; Pfau et al., 2018; Sandler & Lillo-Martin, 2001, 2008).

In parallel to the theoretical relevance of linguistic competence independent of the modality of language use (Chomsky, 1965; van der Burght et al., 2023), early psycholinguistic research began to investigate how abstract linguistic representations might affect ongoing cognitive processes in comprehension. A paradigm that was applied in those research lines was the detection of extraneous sensory signals such as an auditory click embedded in sentences (henceforth, the click paradigm; see Cutler & Norris, 1979, for a detailed overview). In this paradigm, initially introduced by Ladefoged and Broadbent (1960), participants listen to speech signals in which a short click occurs, and are asked to report when it occurred, usually by marking its position (e.g., with a ‘/’) in a written version of sentence they heard, or by typing both what they heard and the click position. We refer to these modality responses as offline responses, as the responses were made after the speech signal was over or generally without time pressure. The click paradigm, in its offline version, was used to investigate the extent to which the linguistic units assumed from a competence perspective could find a correlate in the performance domain. For example, Fodor and Bever (1965) employed this paradigm to investigate “the psychological reality of linguistic segments”, as evidenced in the title of their work, to test whether syntactic constituents might function as perceptual units in speech comprehension. Based on the notion that perceptual units maintain unity despite interfering stimuli, they interpreted the fact that participants mislocated clicks occurring within constituents, “migrating” them towards the closest major constituent boundary, as evidence for constituents being a psychological or perceptual reality in speech perception. Similar results were observed in follow-up work by Garrett and colleagues (1966) and by Bever, Lacker, and Kirk (1969), who further showed that this effect was specific to boundaries between clauses and absent for boundaries between phrases.

Building on the results of Fodor and Bever (1965), Holmes and Forster (1972) further observed that, in the offline click paradigm, not only were migration effects present, but also a modulation of the accuracy according to the click position in the syntactic structure. In particular, they showed that, even independently from migration effects (or with mild migration effects), accuracy was higher for clicks located between clauses, slightly decreasing for clicks between phrases, and becoming even smaller for clicks presented within phrases. This finding converged with the result of a previous work employing an online response modality, in which participants were asked to respond as fast as possible via button press as soon as they heard the click (Holmes & Forster, 1970). In this study, reaction times (RTs) for click detection were the fastest for clicks located between clauses, intermediate for clicks located between phrases, and the slowest for clicks within a phrase. This pattern of results was most strongly observed in the first half of a sentence, while differences between conditions became less pronounced or disappeared in the second half of the sentence. Building on the results of these two studies, Holmes and Forster (1970, 1972) suggest that the click paradigm reflects the processing load during sentence comprehension. Accordingly, both the accuracy (Holmes & Forster, 1972) and RTs (Holmes & Forster, 1970) data would indicate that the processing load is higher within constituents – hence, the lowest accuracy and highest RTs observed when the click occurs in this position – intermediate within phrases, and smaller between clauses. Furthermore, this account of the click paradigm would also be able to account for the fact that RTs were generally faster in the second half of the sentence (Holmes & Forster, 1970), as, in this case, the processing load might be generally lower than in the first part of the sentence, due to the presence of linguistic predictions. Subsequent work observing that the RTs of detecting clicks at a clause boundary were modulated by syntactic complexity (e.g., subject relatives vs. object relatives) further corroborated the notion that the click paradigm can reflect processing load during sentence comprehension (Cohen & Mehler, 1996).

In the present work, focusing on sign rather than spoken language, we test whether the click paradigm, in its online response format, might reveal processing load during the comprehension of German Sign Language (DSG). Accordingly, we assume the interpretation put forward by Holmes and Forster (1970, 1972) linking click detection to current processing load. We, however, point out that the click paradigm has also been adopted to investigate linguistic dimensions beyond hierarchical structure (for an overview, see Cutler & Norris, 1979), and, in some of these instances (i.e., open vs. closed class status), the pattern of results was consistent with modulation driven by acoustic low-level features, rather than high-level linguistic ones (Cutler et al., 1993). At the same time, we believe that the results of Holmes and Forster (1970, 1972) are largely consistent with several studies in the fields of psycholinguistics (e.g., Bever, 1988; Miller, 1964; see Levelt, 1995, for an overview) and neurolinguistics (Dehaene et al., 2022; Friederici et al., 2017; Hale et al., 2022; Maran, Friederici, et al., 2022; see Zaccarella & Trettenbrein, 2021, for an overview), demonstrating the pertinence of hierarchical constituent structure during auditory and written language processing, as well as automatic behavioral (Berkovitch & Dehaene, 2019; Pyatigorskaya et al., 2023) and neural (Batterink & Neville, 2013; Hahne & Friederici, 1999; Pulvermüller & Shtyrov, 2003; Pyatigorskaya et al., 2025) effects driven by syntactic structure.

In contrast to the extensively studied auditory and written domains, the link between abstract syntactic structure and cognitive processing in sign language processing remains poorly unerstood. Research over the past decades has demonstrated that sign languages as well as spoken and written languages are both primarily processed in the brain’s left-hemispheric language network (Campbell et al., 2007; Emmorey, 2015; Trettenbrein, Papitto, et al., 2021), with similar structural connectivity in the left, but not the right, hemisphere, compared to hearing individuals (Trettenbrein, Klein, et al., 2025). On the one hand, electroencephalographic studies with evoked potentials have shown that deaf signers’ brains show sensitivity to grammatical violations similar to the evoked potentials known from studies of spoken and written language processing in hearing non-signers (Capek et al., 2009; Hänel-Faulhaber et al., 2014; Hosemann et al., 2018). On the other hand, functional magnetic resonance imaging studies aiming to identify the neural basis of syntactic processing and hierarchical structure in sign language so far have yielded conflicting results (Matchin et al., 2021; Moreno et al., 2018; for in-depth discussion, see Trettenbrein, Zaccarella, et al., 2025). Psycholinguistic work on sign languages took an interest in syntactic processing in sign language already early on (e.g., Grosjean & Lane, 1977; Klima et al., 1979), but then has often focused on lexical and semantic processing as well as modality-specific phenomena, such as iconicity (Baus et al., 2013; Bosworth & Emmorey, 2010; Carreiras et al., 2008; Gutiérrez-Sigut & Baus, 2021; Thompson et al., 2009). To the best of our knowledge, no recent psycholinguistic studies have investigated how processing during sign language comprehension is modulated by hierarchy or constituent structure.

Building on the results obtained with spoken language (Holmes & Forster, 1970, 1972), the present study is the first to probe whether the processing load is modulated by hierarchical constituent structure during sign language comprehension. Accordingly, we adapted the classical psycholinguistic click paradigm from the auditory-oral modality of spoken languages to the visuo-spatial modality of sign languages. Using short white flashes inserted into videos of DGS sentences as analogues to clicks in the auditory domain, we sought to determine whether signers’ speed of detection of extraneous perceptual signals is modulated by their position within the hierarchical structure of signed sentences. We performed a pre-registered online experiment in which deaf signers watched videos of DGS sentences and had to respond to the white flashes, which could occur at different points in the constituent structure, as quickly as possible via button press. Building on Cutler et al.’s (1993) approach in the spoken modality, to rule out the possibility that any potential observed effects are perceptual, rather than linguistic, in nature, we performed a second independent pre-registered online experiment, using the same stimuli and experimental paradigm, with a group of participants not knowing DGS (i.e., hearing non-signers).

To preview our results, we observed no effect of our syntactic manipulations in either experiment. Additional analyses revealed effects of topicalization and uncertainty for both groups. In Section 4, we provide two potential explanations of the observed findings. On the one hand, continuing to maintain the assumptions of Holmes and Forster (1970, 1972) regarding the click paradigm, we argue that the simultaneous and time-shifted presence of syntax-relevant cues (i.e., hands, mouthings, and non-manuals) in the sign stream might have made it difficult to observe a modulation of processing load by hierarchical structure, compared to the spoken modality tested in previous studies. On the other hand, building on van der Burght et al. (2023), we highlight the point that, while our study was built on the assumption that the click paradigm can reflect the perceptual unity of syntactic constituents and ongoing processing load in comprehension (Fodor & Bever, 1965; Garrett et al., 1966; Holmes & Forster, 1970, 1972), not all studies employing this paradigm support this assumption (Abrams & Bever, 1969; Cohen & Mehler, 1996; Cutler et al., 1993; Reber, 1973). Finally, we further argue that non-signers attend to some non-manual cues (i.e., eyebrow raising), despite their lack of sign language knowledge, because they resemble communicative gestures (Herrmann & Pendzich, 2014; Pendzich, 2020).

The experimental design, hypotheses, planned analyses, as well as stimulus materials were pre-registered independently at the Open Science Framework for both experiments. The full text of the pre-registration, including stimulus materials for the main experiment with deaf signers, can be found here: https://doi.org/10.17605/OSF.IO/N29AZ. The full text of the pre-registration for the control experiment with hearing non-signers can be found here: https://doi.org/10.17605/OSF.IO/9FR25.

In the present study, we set out to probe whether the processing load during comprehension, as measured by RTs in detecting an extraneous perceptual signal (Holmes & Forster, 1970, 1972), was affected by hierarchical constituent structure during sign language processing. We therefore adapted the classical psycholinguistic click paradigm (Fodor & Bever, 1965; Garrett et al., 1966; Holmes & Forster, 1970, 1972), in its online version (Holmes & Forster, 1970), from the auditory-oral modality of spoken languages to the visuo-spatial modality of sign languages. Accordingly, we employed short flashes inserted into videos of DGS sentences as visual analogues to auditory clicks (see Geers, 1978, for a similar methodological approach). Contrary to our hypothesis, we did not observe any effect of our syntactic manipulation in the main experiment with deaf signers. An additional exploratory reanalysis of the data in a Bayesian framework even revealed strong evidence against an effect of Structure in the experiment with deaf signers. In what follows, we will discuss two different explanations that can account for this null finding. First, we continue to maintain the assumption that the click paradigm captures processing load during sentence comprehension (e.g., Holmes & Forster, 1970, 1972) and we highlight how sign languages might provide a linguistic signal which makes comprehension resilient in face of external sensory stimuli. Second, we drop such an assumption and review some conflicting findings in the literature concerning the click paradigm. We then turn to the results of our exploratory analyses, which raise questions about the processing of non-manual components in sign languages by signers and non-signers alike, and about whether the click paradigm might represent a valuable tool for sign-language studies.

Assuming that RTs in the click paradigm, also in its implementation in the visual modality (Geers, 1978), reflect processing load (Holmes & Forster, 1970, 1972), our results suggest that the hierarchical structure of DSG sentences does not modulate this cognitive aspect during comprehension. This raises the question of why, compared to the spoken modality, an extraneous signal sharing the same perceptual modality as linguistic information does not affect processing in sign, as opposed to spoken, languages. Compared to the spoken modality, the complex simultaneous and time-shifted (i.e., different onset and offset points) presence of syntax-relevant cues (i.e., hands, mouthings, and non-manual markers) may make the cognitive processing of the visual signal of sign language more robust against disruption by extraneous signals than the mostly strictly linear speech stream (Figure 4). Thus, multiple articulators might jointly convey information that can be used to optimize ongoing cognitive processing, hence “freeing” resources that, in the current task, could be exploited to detect an extraneous visual signal. To boost the importance of the sequential syntactic information provided by perceived word, Holmes and Forster (1970) controlled their experiment for possible prosodic confounds. Our study attempted to do something similar by using automated motion-tracking on our stimulus clips (see 2.2 above).

Following the logic of Holmes and Forster (1970), the parser is assumed to build up syntactic structure by sequentially integrating incoming words or signs into the constituent structure during sentence comprehension, based on expectations about the continuation of the sentence. Whereas we still concur that this, in principle, is a reasonable assumption, closer inspection of what syntax-relevant information is present at what time in the sign stream, as opposed to the speech stream, makes it clear that the signed signal is much richer (Meier, 2002) and, consequently, provides several affordances that allow the comprehender to maintain optimal performance despite the additional demands imposed by extraneous signals. In other words, the potential disruptive effect on ongoing processing of the extraneous signal (i.e., the flash) at exactly these points in time appears to have been readily compensated for by the syntax-relevant information provided by the other articulators. Along these lines, it has also been argued that the nature of the major articulators of sign languages requires longer transition phases between signs, whereas these transitions are likely already informative with regard to the next sign (Hosemann et al., 2013; Wienholz et al., 2023; also see Grosvald & Corina, 2012). Hence, the different demands on temporal chunking imposed by the visual and auditory modality may lead signers to rely less on serial maintenance of linguistic units (Giezen, 2021).

Interestingly, previous work inspired by the click paradigm showed that visual flashes disrupt lip reading of spoken sentences in deaf signers (Geers, 1978). This work is of both methodological and theoretical relevance for the present findings. On the methodological level, it validates the presentation of flashes to non-hearing signers to disrupt ongoing processing. On the theoretical level, it shows that when extraneous visual signals disrupt communicative information that is serial (i.e., lip movements) and, therefore, not as redundant as signed sentences (i.e., simultaneous effectors), the cognitive load is increased in signers. Therefore, the findings of Geers (1978) are, in principle, compatible with the hypothesis we put forward above. At the same time, we acknowledge that lip reading of spoken sentences might, in itself, pose an increased cognitive load for signers (i.e., the use of a second language and only a proxy of the actual signal being available) as opposed to the processing of signed sentences (i.e., the use of their native language). Accordingly, we believe that our account of the combined results of Geers (1978) and our study might be worth exploring in future studies, comparing the effect of visual flashes on signed sentences and other visual serial communicative information.

In addition to the many-to-one mapping required during sign language comprehension specifically for syntax-relevant cues, the canonical word order of DGS and other languages (sign and spoken) may impact the degree to which predictions of sequential elements constitute a cognitively useful strategy. That is, a central tenet of Holmes and Forster’s (1970) paradigm and, by extension, also of our adaption of this paradigm to the visuo-spatial modality, is that participants’ responses to an extraneous signal will be influenced by what further elements the parser anticipates based on previously encountered elements (i.e., the parser projects and maintains an open node which is reflected by increased processing load). While this is a reasonable assumption on the level of minor and major constituents, such predictive processing may simply not occur on the phrasal level (see Maran, Friederici, et al., 2022; Maran, Numssen, et al., 2022; for discussion), especially in languages where elements such as adjectives or adpositions usually follow the element that they modify (e.g., unlike in English or German, the adjective NEW follows the noun BOOK in DGS, as shown in the example sentence in (4); see Section 1). Accordingly, some trials of the within-phrase manipulation (i.e., the ‘no break’ conditions) in our main experiment may have been affected by this (i.e., the parser does not project an open node after BOOK, instead the structure [BOOK NEW] is only built up if and when a postpositional element is encountered). While the click paradigm has been employed in studies testing spoken languages in which adjectives or modifiers follow, rather than precede, the element they modify (e.g., Italian: van Ooijen et al., 1993; Spanish; Lobina et al., 2018; Mota & Igoa, 2017), they focused on linguistic dimensions other than the ones tested in the present paradigm.

The different points raised in the discussion above rest on the assumption that the click paradigm captures ongoing processing load; however, alternative explanations are possible once this assumption is dropped. That is, while a large number of studies have observed an effect on performance for the positioning of clicks with regard to the syntactic structure of sentences, other studies have questioned the general validity of the paradigm (see also Lobina et al., 2018 for related discussion). For example, Reber (1973) showed that even participants who are instructed to locate a “subliminal click” in a sentence have a tendency to locate it at a constituent boundary, despite the fact no click was actually presented. While this finding is compatible with the idea that constituents are perceptual units in language processing, they are problematic as they suggest that click paradigms might not capture the effect of an extraneous visual signal, but rather an implicit syntactic bias. Furthermore, Cutler and colleagues (1993) showed that the click paradigm might be strongly influenced by low-level properties of the stimulus materials, as similar results can be obtained also in participants without competence in the tested language. While these results (Cutler et al., 1993) were obtained focusing on a different linguistic feature (i.e., the open vs. closed class status of words) than the one tested here (i.e., hierarchical structure), they raise important questions about previous work that adapts this paradigm without ensuring that the effect is present only in participants who know the tested language.

Despite our null finding in the present study, we believe that the linguistic evidence for hierarchical constituent structure in the sign languages of the world is compelling (Cecchetto, 2017; Kocab et al., 2023; Mathur & Rathmann, 2014; Sandler & Lillo-Martin, 2001, 2008; Tang & Lau, 2012). Accordingly, we caution readers not to interpret our failure to observe syntax-specific effects in this study, which uses a comprehension paradigm, as evidence for the lack or irrelevance of hierarchical constituent structure in the comprehension or production of DGS in general. Instead, against the background of the general state of research on constituent structure and hierarchy in language and, specifically, sign language, we believe that either the greater robustness of the sign stream or limitations of the click paradigm provide a reasonable explanation for the observed null result. At the same time, we believe that our findings, and especially the methodological approach we developed for creating stimulus materials, which we make openly available in the dedicated repository, warrant further studies using different variations of the click paradigm (e.g., a version that does not rely on RTs during comprehension, but instead increases the memory demands imposed on participants by requiring them to first encode, and then decode, the perceived sentences; Holmes & Forster, 1972; Seitz & Weber, 1974).

To further explore the data collected, we performed two additional exploratory analyses, which suggested that the presence of non-manual markers for topicalization in some items has a significant effect on the general responses to the “flashes” in both groups of participants, demonstrating that hearing non-signers attend to some non-manual cues of DGS (i.e., raised eyebrows marking topicalized structures), despite their lack of knowledge of the language. Interestingly, the main effect of Topicalization was not just present in the main experiment as well as the control experiment, but also showed the same directionality (i.e., signers and non-signers both were generally slower to respond to topicalized items, regardless of whether the flash occurred in the first or second half of the sentence). Moreover, hearing non-signers’ responses were especially inhibited when the non-manual marker was present at the beginning of stimuli, presumably due to the overlap of this syntactic marker of DGS with a non-manual gesture (Dohen & Lœvenbruck, 2009; Wierzbicka, 2000). Building on Cutler et al. (1993), the fact that the effect is present in both groups rules out an explanation on the grounds of language knowledge. However, the exploratory nature of this analysis does not allow us to determine whether the observed effects are driven by different causes in the two experiments. Despite grammatical and lexical functions of non-manuals in the visuo-spatial modality, both deaf signers and hearing non-signers also use non-manuals to express emotions, reactions, and attitudes (Pendzich, 2020; Wilbur, 2021). Specifically, Pendzich (2020) showed that raised eyebrows presented in isolation, without a signed context, are interpreted as an interrogative marker by deaf signers as well as hearing non-signers, pointing towards a possible gestural origin of this grammatical marker (Domaneschi et al., 2017; Janzen, 1999; Van Loon et al., 2014). Furthermore, this facial cue also showed a strong relation to scepticism for hearing non-signers (Pendzich, 2020).

The finding that signers and non-signers show similar effects for topicalized items, despite the difference in their knowledge of DGS, in combination with the observation that in both experiments, participants were actually slower to respond when non-manual components in the form of raised eyebrows (i.e., topicalization markers) were present in the stimulus, raises questions regarding the cognitive status of non-manuals. In the literature, two different views have been put forward: Some have considered non-manuals as the direct surface realisation of syntax (Emmorey, 2002; Wilbur & Patschke, 1999), whereas others have argued that some aspects of the linguistic analysis of non-manuals may be achieved by other (not necessarily linguistic) cognitive systems (Atkinson et al., 2004), presumably in tandem with the language system. That is, signers are assumed to at least initially recruit additional cognitive resources (e.g., non-linguistic systems for processing facial gestures with a lexicalised meaning, such as ‘top’), but this information is then forwarded to the language system. Our data cannot resolve this question, but they seem to indicate that the use of non-manual components, indeed, imposes additional processing demands, regardless of participants’ knowledge of DGS: Both experiments showed that participants were slower to respond when items included topicalization markers, though only the group of deaf signers could have processed them as such grammaticalised components.

Lastly, the Bayesian reanalysis of the data from both experiments confirmed the null result for our syntactic manipulation, but, at the same time, also revealed evidence for an effect of position, thus indicating that the original effect of position observed by Holmes and Forster (1970) most likely is not linguistic in nature, but instead reflects a general effect of attention. In the original article that served as an inspiration for the present study, Holmes and Forster (1970) concluded that participants’ slower responses to clicks occurring in the first half of the sentence, as opposed to the second half, reflected the decreased processing demands during the second half of the sentence, because more linguistic information has already been encountered, while the kind of new linguistic information that needs to be integrated becomes more predictable. Our adaptation of the paradigm to the visuo-spatial modality of sign language, in combination with the data from the control experiment, allow us reconsider this claim: If the response pattern described by Holmes and Forster was, indeed, only caused by increased cognitive demands during processing complex sentences, we should not have observed an effect of position in our control experiment with hearing non-signers, who did not process the stimuli in a linguistic manner. In contrast, our Bayesian reanalysis showed an effect of Position in both experiments, though more pronounced for the main experiment with deaf signers. Hence, we argue that any observed effect of Position in this paradigm actually does not necessarily reflect decreased linguistic processing demands, but indexes decreasing uncertainty as a trial progresses during the experiment.

While linguistic research has demonstrated that sentences exhibit a complex internal structure independent of modality (i.e., speech or sign), our set of pre-registered experiments presented here, which adapted the classical psycholinguistic click paradigm to the visuo-spatial modality, failed to observe any influence of hierarchical constituent structure on cognitive processing load during sign language comprehension. Interestingly, additional exploratory analyses suggested that the behaviour of non-signers was systematically influenced by non-manual cues, despite their lack of knowledge of DGS. The Bayesian reanalysis of the data from both experiments then revealed a general effect of uncertainty, depending on whether the flash occurred in the first or second half of the video, independent of participants’ knowledge of DGS.

Against this background, we conclude that the simultaneous and time-shifted presence of different syntax-relevant cues (i.e., hands, mouthings, and non-manuals) makes the sign stream robust against disruption by extraneous visual signals. At the same time, our data indicate that non-signers without any knowledge of DGS nevertheless attend to the non-manual cue for topicalization (i.e., raised eyebrows), despite not comprehending its syntactic relevance, possibly because its form resembles a communicative gesture for interrogatives which has been grammaticalized in DGS (Pendzich, 2020). Future work should probe the constituent structure of sign languages using memory-based variations of the click paradigms similar to Holmes and Forster (1972).