UCLA

The synergy between dynamical systems and deep learning (DL) has become an increasingly popular research topic because of the limitation of classic methods and the great potential of DL in addressing these challenges through the data-fitting and feature-extraction power of deep neural networks (DNNs). DNNs have demonstrated their ability to approximate highly complicated functions while enjoying good trainability, which can help dynamical system modeling with both the search of solution and the expressiveness of models. Furthermore, the feature extraction ability of DNNs have proven useful in identifying system states identification when the state cannot be defined from first-principles.

Conversely, the study of dynamical systems has benefited DL. Viewing DNNs as discretization of ordinary differential equations (ODEs) inspires a novel family of models named neural differential equations which offer unique advantages in time series learning, especially the modeling of dynamical systems. From another perspective, viewing the optimization of deep learning models as a dynamical system on the loss landscape enables better analysis and enhancement of the optimization processes.

This work focuses on this interplay. We develop novel deep learning methods to efficiently model dynamical systems, incorporating physical prior knowledge and meta-learning techniques. By analyzing the dynamics of the optimization process, we also design a novel variant of stochastic gradient descent to enhance the resilience of DNNs against weight perturbations, enabling their deployment on analog in-memory computing platforms where analog noise is inevitable. Through these investigations, we contribute to the growing body of research on the intersection of dynamical systems and deep learning, paving the way for innovative solutions to complex real-world problems.