Introduction

Methods

Results

Discussion

A more complete understanding of the biological visual system can inspire the design of computer vision algorithms, and building accurate models constitutes an important step to such an understanding. We utilize computational and deep learning approaches to close the gaps in the literature on modeling the retina and the primary visual cortex (V1), the two important components of the early visual processing pathway.Firstly, to address the lack of a comprehensive computational model of retinal degeneration, we present a biophysically detailed model of the cone pathway in the retina that simulates responses to light and electrical stimulation. Anatomical and neurophysiological changes due to retinal degenerative diseases were systematically introduced. The model was not only able to reproduce common findings about retinal ganglion cell (RGC) activity in the degenerated retina, but also offered testable predictions about the underlying neuroanatomical mechanisms. These insights may further our understanding of retinal processing and inform the design of retinal prostheses. Secondly, to argue for more emphasis on freely moving experimental design, we propose an analysis of the retinal input during free exploration in mice. Mice were able to employ compensatory and gaze-shifting eye-head movements to sample the visual environment during natural locomotion. We found that eye movements preferred features such as edges and textures. A deep learning predictive model of gaze shifts indicated that the upper peripheral visual field contributed most to the prediction, consistent with animal behavior such as predator detection. These results may provide implications for visual processing beyond head-fixed preparations. Lastly, to bridge the gap in predictive modeling tailored for neural data gathered from freely moving experimental paradigms, we introduce a multimodal recurrent neural network that integrates gaze-contingent visual input with behavioral and temporal dynamics to explain V1 activity in freely moving mice. The model achieves state-of-the-art predictions of V1 activity during free exploration. Analyzing our model using maximally activating stimuli and saliency maps, we reveal new insights into cortical function, including the prevalence of mixed selectivity for behavioral variables in mouse V1. Our model offers a comprehensive deep-learning framework for exploring the computational principles underlying V1 neurons in freely-moving animals engaged in natural behavior.

Animals use vision to traverse novel cluttered environments with apparent ease. Evidence suggests that the mammalian brain integrates visual motion cues across a number of remote but interconnected brain regions that make up a visual motion pathway. Although much is known about the neural circuitry that is concerned with motion perception in the primary visual cortex (V1) and the middle temporal (MT) area, little is known about how relevant perceptual variables might be represented in higher-order areas of the motion pathway, and how neural activity in these areas might relate to the behavioral dynamics of locomotion. The main goal of this dissertation is to investigate the computational principles that the mammalian brain might be using to organize low-level motion signals into distributed representations of perceptual variables, and how neural activity in the motion pathway might mediate behavior in reactive navigation tasks. I first investigated how the aperture problem, a fundamental conceptual challenge encountered by all low-level motion systems, can be solved in a spiking neural network model of V1 and MT (consisting of 153,216 neurons and 40 million synapses), relying solely on dynamics and properties gleaned from known electrophysiological and neuroanatomical evidence, and how this neural activity might influence perceptual decision-making. Second, when used with a physical robot performing a reactive navigation task in the real world, I found that the model produced behavioral trajectories that closely matched human psychophysics data. Essential to the success of these studies were software implementations that could execute in real time, which are freely and openly available to the community. Third, using ideas from the efficient-coding and free-energy principles, I demonstrated that a variety of response properties of neurons in the dorsal subregion of the medial superior temporal (MSTd) area could be derived from MT-like input features. This finding suggests that response properties such as 3D translation and rotation selectivity, complex motion perception, and heading selectivity might simply be a by-product of MSTd performing dimensionality reduction on their inputs. The hope is that these studies will not only further our understanding of how the brain works, but also lead to novel algorithms and brain-inspired robots capable of outperforming current artificial systems.