UC San Diego

Transferring information from data to models is crucial to many scientific disciplines. Typically, the data collected are noisy, and the total number of degrees of freedom of the model far exceeds that of the data. For data assimilation in which a physical dynamical system is of interest, one could usually observe only a subset of the vector state of the system at any given time. For an artificial neural network that may be formulated as a dynamical model, observations are limited to only the input and output layers; the network topology of the hidden layers remains flexible. As a result, to train such dynamical models, it is necessary to simultaneously estimate both the observed and unobserved degrees of freedom in the models, along with all the time-independent parameters. These requirements bring significant challenges to the task.

This dissertation develops methods for systematically transferring information from noisy, partial data into nonlinear dynamical models. A theoretical basis for all these methods is first formulated. Specifically, a high-dimensional probability distribution containing the structure of the dynamics and the data is derived. The task can then be formally cast as the evaluation of an expected-value integral under that probability distribution. A well-studied sampling procedure called Hamiltonian Monte Carlo is then introduced as a functioning part to be combined with Precision Annealing, a framework for gradually enforcing the model constraints into the probability distribution.

Numerical applications are then demonstrated on two physical dynamical systems. In each case, inferences are made for both the model states within the observation window and the time-independent parameters. Once complete, the predictive power of the model is then validated by additional data. Following these is a discussion of the role of the state-space representation.

The dissertation concludes with an exploration of new methods for training artificial neural networks without using the well-known backpropagation procedure. Given the equivalence between the structure of an artificial neural network and that of a dynamical system, the aforementioned theoretical basis is applicable in this arena. The computational results presented indicate promising potentials of the proposed methods.