The zebra finch is a social songbird that lives in large groups and produces vocal communicationcalls to facilitate social interactions. A subset of these calls can be used for individual recognition, via distinct acoustic features that are stereotyped within a bird but individualized across birds. Given their large natural group sizes and the ethological importance of individual recognition, one might expect that zebra finches would have the capacity to recognize the calls of a large number of conspecifics. In this thesis, I describe a set of neuroethological experiments to test the memory of zebra finches for individual conspecifics by their vocalizations. We hypothesized that the caudal nidopallium (NCM), a higher-order auditory region of the avian brain analogous to mammalian auditory association cortex, is involved in the learning and retention of these auditory memories. Using an operant task in which birds were trained to associate the calls of some individuals with food reward, zebra finches were found to have a large capacity for recognizing individuals by their calls and song, and that those associations could be learned with just a few training examples and persist for at least a month without reinforcement. Furthermore, lesions to NCM eliminated previously formed associations but did not prevent re-learning or learning of novel stimuli, in contrast with lesions to vocal pre-motor pathways which had no effect on the recognition ability of the birds. Finally, using the spiking activity from single neurons across the cortical-like auditory regions of the brain, we found that familiar and task-relevant vocalizations elicited more reliable neural responses, with higher information capacity, than in response to unfamiliar and less behaviorally relevant calls.

This dissertation investigates the cortical representation of speech perception, using a combination of functional Magnetic Resonance Imaging (fMRI) and psychoacoustical experiments.

Previous research has shown that low-level acoustical structure, phonemes, and words are processed by distinct cortical areas. However, little is known about the relationship between these different representations. To address this problem we simultaneously mapped many different representations of speech. We recorded fMRI responses from subjects listening to over two hours of natural speech. We then examined three features spaces representing the speech sounds in terms of auditory, articulatory and semantic features. We used voxel-wise modeling for each feature space combined with a novel variance-partitioning method to assess how much response variance could be explained uniquely by each model or jointly between two or three models. Validating our approach, we found that a quarter of the brain was significantly responsive to the stories, and that our models could account for up to 45% of the explainable variance in cortex and over 60% of the explainable variance in auditory areas. We also found a hierarchical set of processing steps starting in primary auditory areas and moving along the posteroventral region of the temporal lobe that are involved in the sound to word meaning transformation.

The second part of this dissertation is a psychoacoustical investigation of the modulation power spectrum (MPS) of speech. The MPS is obtained by taking the 2-dimensional Fourier transform of the speech spectrogram. We showed that comprehension of vowels and consonants is differently affected by removal of specific spectral or temporal modulations. Supplementary consonant analysis showed differences in MPS and psychoacoustical comprehension results between three groups of consonants, separated based on the manner in which they are pronounced (fricatives, stops, and sonorants). The MPS could serve as an excellent intermediate step between lower and higher levels of speech processing, and could in future studies add nuance to our previous three cortical models of speech perception.

Natural sounds include animal vocalizations, environmental sounds such as wind, water and fire noises and non-vocal sounds made by animals and humans for communication. These natural sounds have characteristic statistical properties that make them perceptually salient and that drive auditory neurons in optimal regimes for information transmission.

Recent advances in statistics and computer sciences have allowed neuro-physiologists to extract the stimulus-response function of complex auditory neurons from responses to natural sounds. These studies have shown a hierarchical processing that leads to the neural detection of progressively more complex natural sound features and have demonstrated the importance of the acoustical and behavioral contexts for the neural responses.

High-level auditory neurons have shown to be exquisitely selective for conspecific calls. This fine selectivity could play an important role for species recognition, for vocal learning in songbirds and, in the case of the bats, for the processing of the sounds used in echolocation. Research that investigates how communication sounds are categorized into behaviorally meaningful groups (e.g. call types in animals, words in human speech) remains in its infancy.

Animals and humans also excel at separating communication sounds from each other and from background noise. Neurons that detect communication calls in noise have been found but the neural computations involved in sound source separation and natural auditory scene analysis remain overall poorly understood. Thus, future auditory research will have to focus not only on how natural sounds are processed by the auditory system but also on the computations that allow for this processing to occur in natural listening situations.

The complexity of the computations needed in the natural hearing task might require a high-dimensional representation provided by ensemble of neurons and the use of natural sounds might be the best solution for understanding the ensemble neural code.

Animals and humans are able to communicate vocally in very challenging acoustic conditions. Background noise, especially from other individuals of the same species, may mask the relevant signal and propagation can introduce significant distortions to the sound waveform. While our brains are able to extract meaningful information from heavily degraded communication sounds, the mechanisms by which the auditory system performs this task are not well understood. This thesis shows how neural systems can and do handle signal degradations. by examining how auditory neurons in an animal model of communication, the Zebra Finch Taeniopygia guttata, process degraded and undegraded signals, and demonstrates that these principles can be used to perform noise reduction on voice recordings. I discuss how the notion of invariance, common in studies of sensory perception for roughly a century, has more recently been helpful in the analysis of sensory systems at the neural level. I discuss how to characterize the invariant properties of vocal sounds, and how to connect this analysis to the mathematical theory of invariants.

To characterize invariance at the neural level, I construct a novel metric using spike-train cross-correlation between neural responses to the same signal obtained under various conditions. Using this measure, I show that a subset of neurons in avian secondary auditory forebrain area NCM can extract a representation of birdsong that is robust to background noise interference. Spectro-temporal receptive field (STRF) and modulation transfer function (MTF) analysis show that these invariant neurons are sensitive to slowly changing pitch features. Then, using stimuli that have been degraded systematically along spectral and temporal features, I further characterize the nature and origin of invariant response properties in neurons throughout avian auditory forebrain. The response of auditory neurons to spectral degradation is well explained by their MTF, but results in the temporal domain show that some neurons show invariance properties beyond those expected from this model.

Finally, I use the insights from these experiments to construct a noise-reduction algorithm that can be implemented in real-time on digital systems. The system performs well when compared to state-of-the-art algorithms for noise reduction and I discuss how these systems interrelate in terms of processing the statistics of vocal sounds. Using these comparisons and interpretations, I show how we might improve the performance of such noise reduction algorithms.

Although a universal code for the acoustic features of animal vocal communication calls may not exist, the thorough analysis of the distinctive acoustical features of vocalization categories is important not only to decipher the acoustical code for a specific species but also to understand the evolution of communication signals and the mechanisms used to produce and understand them.

Here, we recorded more than 8,000 examples of almost all the vocalizations of the domesticated Zebra finch, Taeniopygia guttata: vocalizations produced to establish contact, to form and maintain pair bonds, to sound an alarm, to communicate distress or to advertise hunger or aggressive intents. We characterized each vocalization type using complete representations that avoided any a priori assumptions on the acoustic code, as well as classical bioacoustics measures that could provide more intuitive interpretations. We then used these acoustical features to rigorously determine the potential information-bearing acoustical features for each vocalization type using both a novel regularized classifier and an unsupervised clustering algorithm. Vocalization categories are discriminated by the shape of their frequency spectrum and by their pitch saliency (noisy to tonal vocalizations) but not particularly by their fundamental frequency. Notably, the spectral shape of zebra finch vocalizations contains peaks or formants that vary systematically across categories and that would be generated by active control of both the vocal organ (source) and the upper vocal tract (filter).

We undertook a detailed analysis of population spike rate and LFP power in the Zebra finch auditory system. Utilizing the full range of Zebra finch vocalizations and dual-hemisphere multielectrode recordings from auditory neurons, we used encoder models to show how intuitive acoustic features such as amplitude, spectral shape and pitch drive the spike rate of individual neurons and LFP power on electrodes. Using ensemble decoding approaches, we show that these acoustic features can be successfully decoded from the population spike rate vector and the power spectra of the multielectrode LFP with comparable performance. In addition we found that adding pairwise spike synchrony to the spike rate decoder boosts performance above that of the population spike rate alone, or LFP power spectra. We also found that decoder performance grows quickly with the addition of more neurons, but there is notable redundancy in the population code. Finally, we demonstrate that LFP power on an electrode can be well predicted by population spike rate and spike synchrony. High frequency LFP power (80-190Hz) integrates neural activity spatially over a distance of up to 250 microns, while low frequency LFP power (0-30Hz) can integrate neural activity originating up to 800 microns away from the recording electrode.

To understand how an auditory system processes complex sounds, it is essential to understand how the temporal envelope of sounds, i.e. the time-varying amplitude, is encoded by neural activity. We studied the temporal envelope of Zebra finch vocalizations, and show that it exhibits modulations in the 0-30Hz range, similar to human speech. We then built linear filter models to predict 0-30Hz LFP activity from the temporal envelopes of vocalizations, achieving surprisingly high performance for electrodes near thalamorecipient areas of Zebra finch auditory cortex. We then show that there are two spatially-distinct subnetworks that resonate at different frequency bands, one subnetwork that resonates around 19Hz, and another subnetwork that resonates at 14Hz. These two subnetworks are present in every anatomical region. Finally we show that we can improve predictive performance with recurrent neural network models.