In Part I of the thesis, we present a body of work analyzing and deriving data-centric regularization methods for the effective training of machine learning models. Machine learning and deep learning in particular have been highly successful in computer vision and generative modelling in recent years. Nonetheless, the progress of such approaches crucially relies on effective regularization, architectural, and algorithmic choices that are often abstracted away during a first consideration. In this part we present the reader with effective regularization approaches focused on the geometry and biases of natural data and parameterization of deep neural networks. We start by deriving a regularization to accurately capture geometric robustness and natural variances of images in Chapter 1. This approach enables significant improvement in model robustness and relies on the theory of optimal transport which we introduce alongside with our method in the chapter. Dataset regularization is extended to active manipulation of the sampling distribution as opposed to each datum in Chapter 2. In the chapter, we present a general and differentiable technique for dataset optimization enabling de-biasing of noisy and imbalanced datasets. In our final contribution for Part I, In Chapter 3, we study the interplay between data and model parameterization. This concerns with the widely-spread architectural approach of neural network normalization. We analyze the convergence dynamics of Weight Normalization and present the first proof of global convergence for dynamically normalized ReLU networks when trained with gradient descent.

In Part II, we study the fluid dynamics phenomena known as the tears of wine problem for thin films in water-ethanol mixtures and present a model for the climbing dynamics. The new formulation includes a Marangoni stress balanced by both the normal and tangential components of gravity as well as surface tension which lead to distinctly different behavior. The prior literature did not address the wine tears but rather the behavior of the film at earlier stages and the behavior of the meniscus. In the lubrication limit we obtain an equation that is already well-known for rising films in the presence of thermal gradients. Such models can exhibit nonclassical shocks that are undercompressive. We present basic theory that allows one to identify the signature of an undercompressive wave. We observe both compressive and undercompressive waves in new experiments and we argue that, in the case of a preswirled glass, the famous “wine tears” emerge from a reverse undercompressive shock originating at the meniscus.