UC San Diego

The study investigates the nature of multimodal communication as a process situated in socially organized and culturally rich environments of practice. To do so it describes ways in which neuroscientists make meaning of the visual fMRI evidence. It examines everyday laboratory practices (e.g., how a novice learns through interaction with an expert and the material world of practice, and how neuroimaging data are analyzed and presented to the larger scientific community), to describe how practitioners engage with digital brain images while they coordinate a variety of multimodal semiotic means such as gesture, talk, gaze, and body orientation. The study involves traditional ethnographic observations and digital video and audio recordings of practices conducted in three laboratories in the Cognitive Science department at UCSD and the Salk Institute. By acknowledging the mutual dependence of communication, signification, and inference the study also advances claims about two theoretical currents in the contemporary cognitive science: embodiment and distributed cognition. First, it questions the supposition that the body is a non-problematic, natural, self-standing phenomenon, commonly assumed in the literature on embodiment. It, also, defines mechanisms of distributed cognition in terms of the multimodal semiotic actions (the multimodal actions not only represent cognitive processes, they participate in, and structure such processes), and it provides evidence for its principles. While MRI practitioners localize cognitive functions in the brain of single individuals, the analysis of their practice reveals that cognition is always relative to the cultural and social world of embodied subjects