UC Davis

With the increasing complexity and dimensionality of data across many fields and applications, traditional data analysis methods fail to properly reveal the insight concealed within the data, made further inaccessible through the ``curse of dimensionality,'' complications inherent to high-dimensional spaces. Motivated by the recognition that high-dimensional data often lie near lower-dimensional structures or manifolds within the high-dimensional space, manifold learning algorithms aim to provide more meaningful and efficient representations by capturing the intrinsic geometry of data, facilitating downstream analysis and visualization of data.

Building on manifold learning and recognizing the fact that there does not exist a universally good representation of data, deep representation learning considers learning parameterized mappings in the biologically inspired form of neural networks and through the mechanism of gradient descent, often referred to as backpropagation. This extends the manifold learning representation capability beyond strictly attempting to recover the low-dimensional geometry of the data and allows for both tailoring the objective to the desired outcome and better introducing inductive biases, which are structures and mechanisms that are designed to prompt the learning system to favor solutions that meet certain priors, assumptions, and presuppositions about the task or data.

In the first part of this thesis, we demonstrate how using manifold learning techniques enables building solutions to problems in complex systems. Specifically, we first show how a class of localization problems in wireless sensor networks can be efficiently tackled by learning low-dimensional diffusion map representations through implicit pairwise relationships, vastly simplifying the original combinatorial optimization problem. Next, we analyze the US census data and show that the features learned through manifold learning can serve as potential multivariate socioeconomic indicators that are robust against partisan manipulation given that the design choices are few and abstract, and in particular we demonstrate interesting consequences of considering one such feature as a deprivation index.

In the second part, we propose algorithms for deep representation learning and demonstrate their competitive performance. The first chapter in this part considers the problem of anomaly detection, where we demonstrate through theoretical analysis, numerical simulations, and experimental benchmarking how a simple modification of the objective and anomaly score function of an autoencoder driven by a basic principle dramatically improves its anomaly detection performance, even beyond considerably more sophisticated algorithms with additional components. Further, we develop an improved scheme for metric few-shot classification within a statistical framework and inspired by human cognition and perception and show its competitive performance. Finally, we incorporate a modified variant of the developed few-shot classifier in a novel description-based navigation system design, which aims at significantly reducing the need for sensors and positioning devices, as well as the need for training in the target environment or exposure to substantial amounts of data from it.