How neurons in the brain collectively represent stimuli is a long standing open problem. Studies in many species from leech and cricket to primate show that animals behavior, such as gaze direction or arm movement, often correlates with a measure of neural activity

termed the population vector. To construct it, one averages preferred stimuli for each neuron proportionately to their responses. However, the population vector discards much of the information contained in the activity of the population. In the first part of this thesis we show that for a broad class of common models of neural populations a sufficient statistic of the population response can be constructed that is guaranteed to transmit as much information about the stimulus as the population response. The statistic has fixed dimension, independent of the population size, and is valid even in the presence of intrinsic interneuronal correlations. This statistic turns out to be a re-weighted version of the population vector. We validate the performance of this statistic on a dataset of visual neural responses. Additionally we show that under certain conditions, this statistic can serve as a reconstruction of the stimulus itself. Quantifying mutual information between inputs and outputs of a large neural circuit is an important open problem in both machine learning and neuroscience. However, evaluation of the mutual information is known to be generally intractable for large systems due to the exponential growth in the number of terms that need to be evaluated. Here we show how information contained in the responses of large neural populations can be effectively computed for a class of models that generalize those considered in the first part of the thesis. Neural responses in this model can remain sensitive to multiple stimulus components. We show that the mutual information in this model can be effectively approximated as a sum of lower- dimensional conditional mutual information terms. The approximations become exact in the limit of large neural populations and for certain conditions on the distribution of receptive fields across the neural population. We empirically find that these approximations continue to work well even when the conditions on the receptive field distributions are not fulfilled. The computing cost for the proposed methods grows linearly in the dimension of the input, and compares favourably with other approximations.

Every day we are immersed in a rich environment of signals to which our nervous system needs to sense and respond. At the sensory periphery, neurons instantaneously respond to stimuli like light and sound, laying the foundation for our perceptual experiences. As we ascend to higher-level brain areas, our cognitive landscape expands, embracing abstract concepts such as space. In this thesis, we unravel the hierarchical organization of neural responses in two specific brain regions, which provided insights into the underlying structure of their neural codes. Additionally, we explored the statistics of natural acoustics and delineated the hierarchies governing their organization.In Chapter 1, we studied spatial representation in hippocampus and demonstrated how neural circuits can achieve efficient representations with dynamic hierarchical organization. In Chapter 2, we studied auditory encoding in inferior colliculus and deciphered the coding scheme used there to support segmental processing of sounds. This involves a hierarchical organization of neural responses reflecting these neurons different response nonlinearities. In Chapter 3, we decomposed natural acoustics into fundamental building blocks and used hyperbolic geometry to capture the correlation statistics among them. The results informed us of the layered structures within various natural acoustics, which also implied a generative model of sounds.

Information maximization has been one of the guiding principles for understanding sensory neural processing. Given the framework, our goal is to explain nonlinear processing by a group of neurons in the retina, encoding the same filter inputs. We begin with a single-cell model, then extend to a neural population subject to relevant constraints, including metabolic costs and neural noise. Still, their predictions only explain part of the observation. Ultimately, we introduce an extra factor, the noise due to modulation, for better elucidating the retina data.Modulation of neuronal thresholds is ubiquitous in the brain. Phenomena such as figure-ground segmentation, motion detection, stimulus anticipation, and shifts in attention all involve changes in a neuron's threshold based on signals from larger scales than its primary inputs. However, this modulation reduces the accuracy with which neurons can represent their primary inputs, creating a mystery as to why threshold modulation is widespread in the brain. We find that modulation is less detrimental than other forms of neuronal variability. Its adverse effects can be nearly eliminated if modulation is applied selectively to sparsely responding neurons in a circuit by inhibitory neurons. We verify these predictions in the retina, where we find that inhibitory amacrine cells selectively deliver modulation signals to sparsely responding ganglion cell types. Our findings elucidate the central role that inhibitory neurons play in maximizing information transmission under modulation.

Many biological systems have intrinsic hierarchical structure, which can be best described by hyperbolic geometry. Hyperbolic geometry was well developed in mathematics, and has recently been introduced in machine learning for data representation and visualization. However, the relevance of hyperbolic geometry to biological data was rarely studied. In this thesis, I systematically detect the existence of hyperbolic geometry in various biological systems, and show the advantages of hyperbolic representation of biological data. The geometry detection of these data are performed using either the fine-but-slow Betti curve method or the rough-but- fast multi-dimensional scaling method. It turns out that many biological systems, including olfactory data from different species and gene expression profiles from different sources, all have hyperbolic structure. The visualization of the data can be achieved by using either hyperbolic multi-dimensional scaling or hyperbolic t-SNE algorithm. The former method correctly reveals the global organization of data points and determines the phenotype related axes in the space, e.g. the pleasantness axis in olfactory space. The latter method aims to perform local clustering as well as preserving inter-cluster structure, giving an accurate representation of different cell types in an informative and organized way. In addition to the improved data visualization, hyperbolic representation of data can inspire new discoveries that cannot be obtained by other Euclidean-based methods, for example, different cell types are characterized by distinct spatial patterns of points in the hyperbolic disk, regardless of brain regions.

Random recurrent networks facilitate the tractable analysis of large networks. The spectrum of the connectivity matrix, determined analytically by random matrix techniques, determines the network’s linear dynamics as well as the stability of the nonlinear dynamics. Knowledge of the onset of chaos helps determine the networks computational capabilities and memory capacity. However, fully homogeneous random networks lack the non-trivial structures found in real world networks, such as cell-types and plasticity induced correlations in neural networks. We address this deficiency by investigating the impact of correlations between forward and reverse connections, which may depend on the neuronal type. Using random matrix theory, we derive a formula that efficiently computes the eigenvalue spectrum of large random matrices with block-structured correlations. The inclusion of structured correlations distorts the eigenvalue distribution in a nontrivial way; the distribution is neither a circle nor an ellipse. We find that layered networks with strong interlayer correlations have gapped spectra. For antisymmetric layered networks, oscillatory modes dominate the linear dynamics.

We analyze the effect of structured correlations on the nonlinear dynamics of rate networks by developing a set of dynamical mean field equations applicable for large system sizes. We find that the power spectrum of strongly antisymmetric bipartite networks peaks at nonzero frequency, miming the gap present in the eigenvalue distribution. Heterogeneous connection statistics facilitate the presence of strongly feed-forward connections in addition to recurrent ones, both of which promote signal amplification. We investigate the role of feed-forward amplification in i.i.d. block-structured networks by computing the Fisher information of past input perturbations. We apply this result to find the optimal architecture for information retention in two populations, under energy constraints. We find that this architecture is both strongly feed-forward and recurrent, with the respective strengths of these connections depending on the available synaptic gain. Finally, we assess the ability of rate networks to dynamically approximate the dominant mode of a random symmetric matrix. Given an initial estimate of the eigenvector as input, we find that there is an optimal processing time and synaptic gain strength depending on the dimensionality and quality of the initial estimate.

The nervous system encodes information about external stimuli through sophisticated computations performed by vast networks of sensory neurons. Since the space of all possible stimuli is much larger than the space of those that are ultimately meaningful,

dimensionality reduction techniques were developed to identify the subspace of stimulus space relevant to neural activity. However, dimensionality reduction methods provide limited insight into the nonlinear functions that build the nervous system’s internal model

of the world. In Chapter 2, the functional basis is introduced that transforms the relevant subspace to a basis that describes the computational function of the subunits that make up the neural circuitry. This functional basis is used to uncover novel insights about

the computations performed by neurons in low-level vision and, later on, high-level auditory circuitry. For the latter, significant barriers are found in the capability of current dimensionality reduction methods to recover the relevant subspaces of high-level sensory

neurons. This barrier is caused by the relative difficulty of stimulating high-level sensory neurons, which are often unresponsive to noise stimuli, while still maintaining a thorough exploration of the stimulus distribution. In response, a new approach to dimensionality

reduction is formulated in Chapter 3 called the low-rank maximum noise entropy method that makes it possible to overcome challenges presented by high-level sensory systems. In Chapter 4, functional bases derived from the relevant subspaces recovered by the

low-rank maximum noise entropy method are employed to study the neural computations performed by high-level auditory neurons.