UC Berkeley

Humans are constantly taking in information from a multitude of dissimilar sources to integrate and update their existing mental representations, but the series of computations that underlie this process are still largely unknown. We do know that these complex processes recruit multiple brain networks that interface with each other, but the precise roles that these networks play can change depending on task demands, and the full extent of this is only partially understood. This research presented in this dissertation focuses on furthering this knowledge specifically in the domain of semantic representation in humans.

In this dissertation, I first summarize the existing literature on semantic representations in the human brain. In addition to this, I describe how careful computational modeling methods alongside naturalistic experimental conditions allow us to answer very precise questions about representations in the human brain. I then present two functional magnetic resonance imaging (fMRI) experiments that explore new avenues of research in semantic representation and further show the strength of these methods. The first of these experiments illustrates that the visual and linguistic semantic networks of the brain are more precisely aligned than had previously been hypothesized. This pattern would not have been discovered using most typical fMRI analysis methods. In the second experiment, I analyze how linguistic semantic representations update after learning and memorization. Specifically, after participants have spent many hours memorizing songs, we can see that their semantic representations of the content of the song lyrics can shift dramatically.

Together, these two experiments show the importance of studying semantic representations not only in a single modality or setting. Without looking at the relationships between different types of representations or how representations are updated with learning and experience, we will never have a full picture of how the brain is functioning in complex, naturalistic environments.