Iconic co-speech gestures are spontaneous body movements produced in coordination with speaking. Concurrently with the word platter, for example, a speaker might trace an oval in the air, showing something about the shape of the object that he is conceptualizing. Chapter 2 investigates fundamental cognitive processes mediating gesture comprehension by comparing ERPs elicited by contextually congruent and incongruent gestures. Because gestures are not part of a conventionalized symbolic system, researchers have argued that they do not convey substantive content on their own; rather, their meaning is driven by speech that accompanies them. Chapter 3 tests these claims by measuring real time semantic activations prompted by iconic gestures presented in the absence of supporting context. To assess how gestures affect comprehension of discourse, EEG was recorded as healthy adults viewed short segments of spontaneous discourse involving both gestures and verbal utterances (Chapter 4). Discourse segments were followed either by related picture probes, which corresponded with information made available both in speech and gesture (Cross-modal Matches), or in speech alone (Speech-only Matches), or by unrelated controls. By comparing brain response to Cross-modal and Speech-only Matches, it is possible to assess the specificity of semantic activations during the integration of speech and gestures. Finally, Chapter 5 investigates whether iconic gestures engage object recognition processes implicated in the comprehension of conventionally depictive representations, such as photographs. "Static gestures" were created by extracting from each dynamic gesture a single freeze frame which made visible critical information about the speaker's intended meaning. EEG was recorded as participants viewed static and dynamic gestures, as well as photographs of common objects. The distribution and time course of ERP effects elicited by these stimulus types were compared

The importance of verbal and visuospatial working memory(WM) for co-speech gesture comprehension was tested in twoexperiments using the dual task paradigm. Healthy, college-aged participants encoded either a dot locations in a grid(Experiment 1), or a series of digits (Experiment 2), andrehearsed them as they performed a discourse comprehensiontask. The discourse comprehension task involved watching avideo of a man describing household objects, and judgingwhich of two words probes was most related to the video.Following the discourse comprehension task, participantsrecalled either the verbally or visuo-spatially encodedinformation. In both experiments, performance on thediscourse comprehension task was faster when gesturalinformation was congruent with the speech than when it wasincongruent. Moreover, performance on the discoursecomprehension task was impacted both by increasing the loadon the visuospatial WM system (Experiment 1) and the verbalWM system (Experiment 2). However, in both studies effectsof WM load and gesture congruency were additive,suggesting they were independent.

Working memory (WM) models have traditionally assumed at least two domain-specific storage systems for verbal and visuo-spatial information. We review data that suggest the existence of an additional slave system devoted to the temporary storage of body movements, and present a novel instrument for its assessment: the movement span task. The movement span task assesses individuals' ability to remember and reproduce meaningless configurations of the body. During the encoding phase of a trial, participants watch short videos of meaningless movements presented in sets varying in size from one to five items. Immediately after encoding, they are prompted to reenact as many items as possible. The movement span task was administered to 90 participants along with standard tests of verbal WM, visuo-spatial WM, and a gesture classification test in which participants judged whether a speaker's gestures were congruent or incongruent with his accompanying speech. Performance on the gesture classification task was not related to standard measures of verbal or visuo-spatial working memory capacity, but was predicted by scores on the movement span task. Results suggest the movement span task can serve as an assessment of individual differences in WM capacity for body-centric information.

Working memory (WM) models have traditionally assumed at least two domain-specific storage systems for verbal and visuo-spatial information. We review data that suggest the existence of an additional slave system devoted to the temporary storage of body movements, and present a novel instrument for its assessment: the movement span task. The movement span task assesses individuals' ability to remember and reproduce meaningless configurations of the body. During the encoding phase of a trial, participants watch short videos of meaningless movements presented in sets varying in size from one to five items. Immediately after encoding, they are prompted to reenact as many items as possible. The movement span task was administered to 90 participants along with standard tests of verbal WM, visuo-spatial WM, and a gesture classification test in which participants judged whether a speaker's gestures were congruent or incongruent with his accompanying speech. Performance on the gesture classification task was not related to standard measures of verbal or visuo-spatial working memory capacity, but was predicted by scores on the movement span task. Results suggest the movement span task can serve as an assessment of individual differences in WM capacity for body-centric information.

Multi-modal discourse comprehension requires speakers to combine information from speech and gestures. To date, little research has addressed the cognitive resources that underlie these processes. Here we used a dual task paradigm to test the relative importance of verbal and visuo-spatial working memory in speech-gesture comprehension. Healthy, collegeaged participants encoded either a series of digits (verbal load) or a series of dot locations in a grid (visuo-spatial load), and rehearsed them (secondary memory task) as they performed a (primary) discourse comprehension task. The latter involved watching a video of a man describing household objects, viewing a picture probe, and judging whether or not the picture was related to the video. Following the discourse comprehension task, participants recalled either the verbally or visuo-spatially encoded information. Regardless of the secondary task, performance on the discourse comprehension task was better when the speaker’s gestures were congruent with his speech than when they were incongruent. However, the congruency advantage was smaller when the concurrent memory task involved a visuo-spatial load than when it involved a verbal load. Results suggest that taxing the visuo-spatial working memory system reduced participants’ ability to benefit from the information in congruent iconic gestures.