UC Berkeley

Understanding the representation of natural stimuli in the human brain is one of the fundamental goals of neuroscience. In this dissertation, I describe an unsupervised learning approach for investigating BOLD (blood oxygen level dependent) responses throughout the cortex to natural movies and speech. This approach for data-driven learning of representational features is particularly useful for regions of the brain where the features aren't well understood.

First, I describe Python software for canonical correlation analysis called Pyrcca, developed for implementing this approach. Then, I introduce a method called FISCA (functional inter-subject component analysis). FISCA uses canonical correlation analysis to estimate components of BOLD responses based on cross-subject similarity. FISCA can be used for combining data across subjects according to functional and not anatomical similarity. FISCA predictively models BOLD responses to novel stimuli across subjects and allows accurate identification of stimulus timepoints based on these predictions.

FISCA components are interpretable. For both natural movie and natural story experiments, I visualize the components on the cortical surface to characterize the role of each component in explaining representation in different brain areas. I also compute the tuning of each component for low-level visual and semantic features. FISCA components are stimulus-agnostic. I demonstrate how FISCA can be used to identify amodal regions by combining data between natural movie and speech experiments.

FISCA is an accurate, flexible, and extensible unsupervised method for modeling brain responses to natural stimuli. Using this approach, I uncover representation of natural visual and speech stimuli throughout the cortex. I hope that in the future, this approach will be used for creating more accurate models and improving our understanding of cortical processing.