UC Berkeley

Despite progress in understanding the organization and function of neural sensory systems, fundamental questions remain about how organisms convert visual, auditory, and other sensory input into useful representations to understand the world and guide behavior. An important and fruitful line of work models the brain as an unsupervised statistical learner, examining how a sensory system may optimize for efficient representation of the natural environment or for explicit representation of useful structure in that environment. This dissertation explores efficient coding and sparse coding models of the visual and auditory systems, the data these systems process, and how these models are affected by the constraints imposed by implementation in biological neural systems. First, I show that both natural images and natural sounds have statistical structure amenable to a sparse coding model but that the sparse structure of these two types of natural data also differ in interesting ways that may be relevant to extending the success of sparse coding in describing primary visual cortex (V1) to analogous regions of the auditory system. I also discuss how a related model may shed light on how the neurons in these sensory systems are organized in space based on coding for related stimulus properties. Second, I show that a sparse coding model with biological constraints requires its inputs to be whitened in order to learn sparse features using synapse-local learning rules. This observation provides a novel explanation for the separation of sparse coding and spatial decorrelation into, respectively, V1 simple cells and preceding areas including retina. Third and finally, I turn back to the auditory system and extend existing work on efficient coding in the cochlea to account for the requirement of causality, i.e., determining a code without knowledge of the future of a signal.