Proceedings of the Annual Meeting of the Cognitive Science Society

Co-speech gesture supports communication, but not everyone uses information from gesture to clarify meaning of spoken messages. Here, we ask whether variability in speech-gesture integration can be explained by visual attention to gesture and propensity to see movement as meaningful gesture. Adults (N=56) viewed videos of a speaker producing statements that were ambiguous or non-ambiguous with respect to a subsequent prompt, with or without meaningful gestures while visual attention was monitored. Presence of gesture significantly increased proportion of correct responses to prompts. However, accuracy was still significantly higher for prompts after non-ambiguous statements, suggesting not all participants used gesture to answer prompts. Explaining some of this variability, when speech was ambiguous, proportion of time spent fixating on gesture-related regions predicted accuracy, a proxy for successful speech-gesture integration. In contrast, although participants who interpreted movement as meaningful gesture were more likely to fixate to gesture, this measure did not predict integration.