- Main
Variational Approaches to Path Problems in Deep Learning
- Psenka, Michael
- Advisor(s): Krishnapriyan, Aditi
Abstract
Many problems in modern deep learning can be framed as path problems: starting from an initial state and given a learned model of the domain, find a path to a target state. This framing recurs across reinforcement learning (an agent transitioning to a high-reward state under a learned critic), molecular modeling (a protein transitioning between conformations under a learned generative model), and long-horizon planning and control (state-action sequences under a learned world model). These problems are typically under-determined: there are infinitely many candidate paths, and the learned model has varying levels of confidence in-between start and target states. A well-posed treatment requires a principled criterion for which paths are best.This dissertation argues that variational reformulations are a useful unifying tool across these problems, and identifies two complementary sub-programs within the broader variational program. The first applies when the data manifold is smooth, low-dimensional, and recoverable: a geometric flow reshapes the manifold toward convexity, which we read as descent on a convexification energy, until the learned representation becomes amenable to classical linear analysis. The second, where the bulk of the dissertation sits, applies when the data manifold (and the models that fit it) can be more rough. Here, we frame path problems as the minimization of a functional on path space induced directly by the learned model itself (its score, its critic, or its dynamics), drawing on classical literature in the calculus of variations and optimal control. Three instances are developed: Q-score matching for diffusion reinforcement learning, Onsager–Machlup interpolation for molecular and image-latent dynamics, and parallel long-horizon planning over world models. For each of these examples, we will see various ways of smoothing out optimization problems over deep learning-based dynamics, whether it be simply through the implicit smoothing-by-splitting of the variational approach or by directly reshaping the gradient structure.