UC San Diego

Songbird vocalizations are complex in nature and rich in information. Parametrizing such high-dimensional signals and extracting embedded information is an important yet difficult task. We approach this problem from three unique angles, incorporating modern state-of-the-art computational tools such as machine learning. We first explore the possibility of characterizing birdsongs with neural activities during song production. We use a recurrent neural network to parametrize zebra finch songs from past spiking activities in HVC. We show that the high-quality song reconstruction is a direct result of the recurrent neural network. While the neural network excelled at learning high-dimensional data, we realize that the distance function commonly used on birdsongs is neither perceptually accurate nor robust to local perturbation. As a solution, we propose the auditory perceptual distance, a computational distance function that characterizes animal vocalizations with acoustic features learned by a convolutional neural network. By training the network on data collected from behaving European starling, we argue the distance function is not only perceptually accurate and robust to local noises, but also highly tunable to a user’s data. Lastly, we seek to better understand the acoustic features used by songbirds, specifically European starlings, to achieve singer recognition. Through both supervised and unsupervised machine learning techniques, we prove vocal textures, characterized by summary statistics, carry a significant amount of singer information and can potentially be used as a vocal signature. By probing trained starlings with familiar textures in behavior experiments, we verify their capability of recognizing familiar singers through their vocal textures. In conclusion, this thesis explores different various ways of characterizing songbird vocalizations to extend our understanding of birdsong production and perception. The pipelines used can also be easily transferred to other species’ vocalizations and has many practical applications.