UC San Diego

Despite the sense of a rich and complete visual world, we are only capable of processing a small fraction of the sensory information available to us. It is no wonder then, that achieving this level of perception involves a series of complex operations from the moment visual information first hits the retina to its eventual interpretation in higher regions of the brain. In this dissertation I explore how visual and auditory information might be leveraged by the visual system to predict impending stimulation and alleviate this burden. In chapter 1, I investigate how the contents of visual working memory captures attention. In a series of studies, I demonstrate that this automatic co-opting of attention is driven by the fidelity of the internal representation, that this capture occurs even when working memory is taxed, and that the quality of any given internal representation is driven by a natural and stochastic accumulation of noise in the representation itself. In the following chapters I explore how real-world auditory objects and scenes facilitate (chapter 2) and alter visual perception (chapter 3). Within these chapters I employ a visual discrimination task where I control the unraveling of visual information to effectively “slow down” visual processing. In chapter 2, I demonstrate that people are able to discern meaningful information more rapidly when the auditory objects and scenes are semantically related to their visual target compared to when they are not. In chapter 3, I demonstrate that this seemingly facilitatory process is not cost free. When the visual targets are less discrete, and more ambiguous, auditory information not only accelerates visual processing but, in so doing, alters the perceptual representation of visual objects, shifting them towards expected features associated with the accompanying sounds. Importantly, this perturbation is not driven by decisional processes, nor is it driven by volitional attentional selection. In summary, this dissertation advances our understanding of visual perception by elucidating the interplay between contextual information across modalities and visual perception. By revealing how auxiliary information influences visual attention and perception, it provides insights into the mechanisms underlying the complex process by which we make sense of the visual world.