Data assimilation transfers information from observations of a complex system to physically-based system models. Typically, the observations are noisy, the model has errors, and the initial state of the model is uncertain, so the data assimilation is statistical. In statistical data assimilation one evaluates the conditional expected values, conditioned on measurements, of interesting quantities on the path of a model through observation and prediction windows. This often requires working with very high dimensional integrals in discrete time descriptions of the observations and model dynamics, which become function integrals in the continuous time limit. Two familiar methods for performing these integrals include (1) Monte Carlo calculations and (2) variational approximations using the method of Laplace plus perturbative corrections to the dominant contributions. We attend here to aspects of the Laplace approximation and develop an annealing method for locating the variational path satisfying the Euler- Lagrange equations that comprises the major contribution to the integrals. This begins with the identification of the minimum action path starting with a situation where the model dynamics is totally unresolved in state space, and the consistent minimum of the variational problem is known. We then proceed to slowly increase the model resolution, seeking to remain in the basin of the minimum action path, until a path that gives the dominant contribution to the integral is identified. After a discussion of some general issues, we give examples of the assimilation process for some simple, instructive models from the geophysical literature. Then we explore a slightly richer model of the same type with two distinct time scales. This is followed by a model characterizing the biophysics of individual neurons.

Characterizing the many physical properties of biological neurons has been historicallydifficult. The use of conductance models describing how properties of these systems change with time can be combined with measured voltage traces to help characterize immeasurable properties of a neuron. This is accomplished using data assimilation, which formulates the inference of these properties as probability maximization using nonlinear optimization. Because measurement noise and model error are inevitable in the study of complex systems, the methods used in this dissertation are designed to cope with unknown processes. This dissertation starts with an overview of data assimilation by formulating the problem xiii data assimilation attempts to solve. I then introduce two methods, nudging and variational annealing, which I use to characterize the properties of a neuron in the Zebra Finch songbird system, HVCX. A key result from this experiment is that there are statistical differences in estimated model parameters for neurons from different birds. Current artificial neural network models are unmanageably large (billions of parameters) and need massive amounts data and computational power to train. The insect olfactory system is a biological network which is capable of learning and identifying new odors quickly in the presence of interference. These properties make it attractive model to explore as a classifier for machine learning tasks. I give an overview of the insect olfactory system by describing its key properties. I use these properties to create a simplified classification system using a winnerless competition network as a pre-processor to a support vector machine. I demonstrate this networks classification capabilities using classes designed to resemble the stimulus an insect receives in the presence of odors. One key result from this experiment is that our network is capable of identifying a mixtures of previously seen odors.

Dynamical systems are an incredibly broad class of systems that pervades every field of science, as well as every aspect of daily life. Not only are they pervasive, but they often exhibit complex behavior resulting from microscopically simple interactions. Examples of such systems are the weather, our brains, animal population, and even home prices – to name but a few. As such, predictions of these systems pose a towering yet necessary challenge, and this work aims at making at dent in this effort. To the extent that models are available, they are useful in that constraints are automatically satisfied, and a mechanistic understanding of the system naturally follows. Part one of this work addresses this case by leveraging our knowledge of the physical model. However, it is often the case that the model is not known, so an effective surrogate model is desired. Part two proceeds in this vein, where the availability of large amounts of data is utilized in constructing surrogate models. Though the theory of dynamical systems still applies to the constructed surrogate model, this approach disregards the physics of the underlying system and has a machine learning flavor.

Complex systems are found at the heart of the most pressing intellectual challenges of the 21st century. Weather and climate, ecological and neurobiological networks, many body quantum systems, and even the operation of human societies can be described as the interaction of simple components from which collectively emerge distinct phenomena. In this thesis, key concepts from dynamical systems theory and machine learning are employed to analyze complex systems in both neurobiological contexts and weather prediction. The first of these concepts is synchronization, from which data assimilation is derived in order to describe the optimal interpolation between an imperfect physical model and sparse/noisy data. Data assimilation is applied to biophysical problems such as the design of VLSI circuits for neuromorphic computing, as well as the estimation of a biological neuron's physical characteristics from voltage and calcium fluorescence measurements. Generalized synchronization is then deployed as the framework to examine the operation of recurrent neural networks, particularly the reservoir computing (RC) architecture. RC operates through the synchronization of a high dimensional artificial network together with chaotic input data, and is able to produce state of the art forecasting performance on a number of benchmark systems in numerical weather prediction. Invariant quantities such as the Lyapunov exponent spectrum and the fractal dimension of the RC are calculated and shown to be directly related to the input dynamics of the data. The concluding remarks unify both RC and data assimilation together into a machine learning forecasting cycle that produces forecasts from sparse and noisy data with no physical model.