UC Santa Cruz

Gestures can both enhance and modify memory for speech when produced alongside it. Although much research has documented the beneficial effects of gesture, far less work has examined the boundaries of the benefits of gesture as well as the mechanisms by which it influences memory. The following series of experiments aimed to understand how and when gesture and speech are represented in memory in an attempt to construct a foundation for how gesture influences what listeners remember. The conditions under which gesture is coactivated during the retrieval of speech were investigated by measuring subsequent memory for gesture across six experiments. In each experiment, participants watched videos of an individual saying brief statements and producing gestures followed by a test on what was said for half of these statements before finally being tested on their memory for gestures themselves. Gesture and speech were said to form an integrated representation in memory in cases where there was an observed improvement in recall of gesture after retrieval of speech.

Overall, these experiments suggest that gesture and speech are coactivated during the retrieval of speech and form an integrated representation in memory. Results provided evidence that such coactivation and thus integration by demonstrating a greater enhancement in memory for gesture after the retrieval of speech when gesture and speech are meaningfully related and irrespective of whether gesture is redundant with the contents of speech. The results also showed that the coactivation of gesture and speech during the retrieval of speech is episodic in nature, implying that the representation of gesture and speech in memory retains episodic details of the experience of watching a speaker talk and move. The experiments presented here help us to better understand how gesture and speech are represented in memory and how such representation may lead to the influence gesture has on memory for speech by directly assessing memory for gesture, when gesture and speech are coactivated, and what processes in retrieval maximally encourage such coactivation.