The rising computational and memory demands of machine learning models, particularly in resource-constrained edge-device settings, motivate us to develop compressed models that reduce both training and inference costs. In this dissertation we develop several algorithms that advance the compressed training capabilities of low-rank tensor models and the uncertainty quantification capabilities of general compressed nonparametric models. First we introduce compressed low-rank tensorized training with rank reduction for the Tensor-Train format. This enables compressed tensorized training with further compression during the training phase. Then we improve our approach in two ways. First we improve scalability by introducing a variational inference algorithm tailored to the tensor learning problem and demonstrate its effectiveness compressing larger-scale models. Second we improve the generality of our approach by extending our tensorized training with rank reduction to all popular low-rank tensor formats. Finally, we introduce a general-purpose algorithm for compressed nonparametric Bayesian learning that can automatically determine the appropriate complexity of nonparametric distributional representations. This algorithm improves the computation/accuracy and storage/accuracy Pareto frontiers over state-of-the-art Bayesian learning approaches.

The photoactivity of a series of vitamin K and vitamin B derivatives was investigated systematically by combining theoretical and computational modeling methods with empirical evidences and experiments. The photochemical properties of lipophilic vitamin K derivatives, including vitamin K1, K2, K3, and K4, water-soluble vitamin K3, as well as water-soluble vitamin B2 derivatives including riboflavin and flavin mononucleotide, are the examples discussed in this work. Some of the vitamins are proven to efficiently generate oxidative triplet excited state (T1) and reactive oxygen species (ROS), such as hydroxyl radicals (HO·), hydrogen peroxide (H2O2), and singlet oxygen (1O2), via the intersystem crossing (ISC) process followed by the type I or type II photoreaction under proper photoirradiation conditions. The generated active species are non-selectively and efficiently bio-lethal to diverse microorganisms including Gram-positive and Gram-positive bacteria, as well as viruses. Meanwhile, some vitamins exhibit excellent photoinduced antimicrobial durability while maintaining their robust antimicrobial function under long-term photo exposure, representing the good photosensitizer nature of these vitamins. In this dissertation, chapter 1 introduces the basic photochemistry principles and photoreaction mechanisms of the aromatic ketone, the parent structure of vitamin K derivatives. Representative inorganic and organic photosensitizers are then identified and discussed. Meanwhile, some natural photosensitizers that can be extracted from natural products are screened. The photoactivity and photochemical properties of some common vitamins are presented as well. In Chapter 2, the photochemistry of the lipophilic vitamin K1-4 is discussed in detail by employing the Gaussian computational modeling package. Vitamin K3 was proven the robust photosensitizer in generating either HO· or 1O2 via type I or type II photoreaction among four vitamin K derivatives. In the following experiments, vitamin K3 demonstrated its highly efficient photoinduced antibacterial function and excellent durability against both Gram-negative and Gram-positive bacteria under daylight irradiation. The successful combination of the theoretical modeling and experiments provides a solid platform to study the photochemistry of photosensitizers at the molecular level. In Chapter 3, vitamin K1, vitamin K3, or vitamin K4 were blended with hydrophobic polyacrylonitrile (PAN) or hydrophobic poly (vinyl alcohol-co-ethylene) (PVA-co-PE) and electrospun to nanofibrous membranes with fibrous diameters of 200-290 nm. The VK3/PVA-co-PE nanofibrous membrane was proven to be the most efficient photoactive nano-membrane in generating ROS under daylight irradiation. The VK3/PVA-co-PE nanofibrous membrane exhibits robust and non-selective microbicidal performance against Gram-negative Escherichia coli bacteria, Gram-positive Listeria innocua bacteria, T7 bacteriophage virus, and Feline Infectious Peritonitis coronavirus under daylight irradiation. Excellent photoinduced antimicrobial durability of the VK3/PVA-co-PE nanofibrous membrane was proven, which confirms the conclusion in chapter 2 and indicates that the application potential of VK3 as a daylight-induced antimicrobial agent was promising. In chapter 4, the photoactivity of a water-soluble vitamin K3, menadione sodium bisulfite (MSB), is discovered and discussed in detail for the first time. It was proven that the triplet MSB could efficiently inactivate bacteria even in the absence of hydrogen donors under UVA (365 nm) irradiation. The determinant factor that affects the photoinduced antibacterial function of MSB is the distance between the photoactive site and the target microorganisms. Excellent photoinduced antibacterial durability of MSB was observed, indicating the photochemical reaction cycles of vitamin Ks with the aromatic ketone structure. In chapter 5, the photochemistry of two water-soluble vitamin B2 derivatives, riboflavin (RF) and flavin mononucleotide (FMN) was studied and discussed. Both RF and FMN were found to produce H2O2 and 1O2 via type I or II photoreaction due to the formation of the oxidative T1. However, RF and FMN show limited photoinduced antibacterial function against bacteria under cool white (370-750 nm) or UVA irradiation. The RF/PVA-co-PE and FMN/PVA-co-PE nanofibrous membranes exhibit good photoinduced antibacterial function against bacteria under UVA irradiation, which is probably due to the intimate contact of the photoactive nanofibrous membranes with target bacteria. Therefore, a close bacterial contact should be provided to RF or FMN to achieve better photoinduced antibacterial performance. The final chapter concludes the photochemistry and photoactivity of vitamin derivatives and their photoinduced antimicrobial function, which may be applied in the fabrication of safe-to-use, highly efficient, extremely durable, and environmentally friendly photoinduced antimicrobial materials that can be applied in bioprotection and food safety related areas.

Training end-to-end models for solving partial differential equations (PDEs) using deep learning methods, such as deep neural networks, demands substantial computing resources, including power supply, memory space, and advanced computing platforms. However, edge devices typically lack these resources, making such training paradigms unattainable on these devices. Transfer learning, which involves leveraging pre-trained models on one dataset to perform inference on another dataset by fine-tuning model parameters, offers a solution by extensively pre-training models on modern GPUs and requiring fewer computing resources during fine-tuning. By applying transfer learning to solve PDEs with neural net-works, we address the demand for real-time response from PDE solvers in scientific and engineering problems. In this project, we propose utilizing transfer learning for Physics-Informed Neural Networks (PINNs) to address problems in reachability analysis. We first pre-train a modified PINN on standard GPUs and subsequently fine-tune the model with constrained computing resources. During fine-tuning, we compute gradients of the loss analytically to reduce dependency on existing libraries, thereby enhancing the method’s generalizability across other edge devices. Through experimentation with multiple PDE examples and the reachability problem, our results demonstrate that transfer learning with limited computing resources achieves comparable accuracy levels to the end-to-end training paradigm while requiring significantly fewer computing resources.

Deep neural networks have demonstrated significant achievements across various fields, yet their memory and time complexities present obstacles for implementing them on resource-constrained devices. Compressing deep neural networks using tensor decomposition can decrease both memory usage and computational costs. The performance of a low-rank tensorized network depends on the choices of hyperparameters including the tensor rank and geometry. Previous studies have concentrated on identifying optimal tensor ranks. This thesis studies the effect of tensor geometry used for folding data for low-rank tensor compression. It is demonstrated that tensor geometry significantly affects compression efficiency of the tensorized data and model parameters. Consequently, a novel mathematical formulation is developed to optimize tensor geometry. The tensor geometry optimization model is adopted for efficient deployment of low-rank neural networks. The presented tensor geometry optimization model is combinatorial and thus challenging to solve. Therefore, surrogate and relaxed versions of the model are developed and various methods including integer linear programming, graph optimization, and random search algorithms are applied to solve the presented optimization model. The proposed tensor geometry optimization achieved a notable reduction in both the memory and time complexities of neural networks while maintaining accuracy. The developed methods can be applied for hardware-software co-design of artificial intelligence (AI) accelerators particularly on resource-constrained devices.

In multimodal sentiment analysis, a model learns to predict one’s opinion towards a certain entity after observing data from multiple sources. As humans merge information from all five senses to read someone,the model needs a proper fusion mechanism to utilize the data from multiple sources. Tensor fusion networks have been proposed and have shown to perform well for multimodal sentiment analysis. However, it suffers from the curse of dimensionality because they use the full tensor format. As a solution, low-rank fusion networks have been proposed, but they require us to pre-determine the rank, which is an ill-posed problem. Instead, we propose adaptive-rank fusion networks that can learn the rank in a data-driven way. We compare the three models in terms of accuracy, complexity, and runtime. Then, we test how adaptive-rank fusion networks react to being trained with a subset of the original training dataset.

In the twilight of Moore’s Law, the semiconductor industry grapples with escalating challenges driven by growing process variations and inherent uncertainties in the manufacturing of increasingly miniaturized devices. Traditional electronic design automation (EDA) tools attempt to address this through rigorous simulation, modeling, and optimization. Concurrently, the need for alternative computational paradigms, notably quantum computing, has surged, introducing fresh complexities and challenges. This dissertation primarily addresses the high cost and low efficiency of modeling and simulation, and the difficulties in design optimization under conditions of noisy and expensive simulations in the context of classical circuits and quantum computing. It proposes uncertainty-aware and efficient design automation methodologies in the EDA field andquantum computing.

The first part of this dissertation focuses on the circuit-level uncertainty quantification and optimization of classical circuits. We first propose a tensor regression model as the stochastic circuit performance model under the high-dimensional process variations. Then, we propose a yield-aware stochastic programming approach to optimize the circuit design, balancing the circuit yield and performance. To tackle optimization in the face of shifted variations, we introduce a distributionally robust optimization formulation, which can be efficiently solved using a Bayesian optimization solver.

The second part addresses block-level simulation and optimization of quantum algorithms. Highlighting the difficulty of the simulation, we begin with an exploration of a variational method employed in the classical simulation of quantum algorithms. Given the substantial obstacle noise poses to the practical applications of quantum computing, we extend the idea of distributionally robust optimization to optimize the parameters of variational quantum algorithms, aiming to enhance the reliability of a quantum algorithm under real-time noise.

This dissertation investigates the challenges of improving the robustness and fairness of deep neural networks through the lens of optimal control theory. Deep neural networks, despite their extensive application across various engineering fields, are vulnerable to imperceptible perturbations and often exhibit biased performance towards underrepresented demographic populations. The first part of this dissertation presents a novel self-healing framework designed to improve the robustness of deep neural networks against unforeseen perturbations. This framework is realized by a novel closed-loop control approach grounded in optimal control theory, which adaptively generates control signals to identify and correct potential errors in the state trajectory of perturbed input data during inference. The second part of this dissertation introduces a PID control framework that generalizes the closed-loop method with additional integral and derivative controllers. We derive an analytical solution for fast online inference, making our control framework applicable to large-scale models. The third part of this dissertation addresses the fairness issue in machine learning within dynamic environments, where undesired model biases against minority users could lead to significant user churn, thereby diminishing the training data for model tuning in subsequent time steps. This negative feedback loop can further exacerbate demographic disparity. To address this, we introduce the concept of asymptotic fairness to maintain consistent model performance across all demographic groups and propose an optimal control solution to achieve this goal.

Deep learning has achieved remarkable success across various domains, ranging from computer vision to healthcare. General-Purpose Graphics Processing Unit (GPGPU) is one of the major driving forces behind this revolution. GPGPUs offer massive parallel computational power, enabling the training and deployment of large-scale neural networks within practical time and resource constraints. Their programmability also enables adaptability to emerging network architectures. However, entering the post-Moore’s Law era, the scaling of computational power offered by GPGPUs struggles to meet the demands of novel neural networks. On the other hand, existing GPGPUs face under-utilization challenges despite the computation power shortage.

This dissertation addresses the computation power shortage by improving the utilization of GPGPUs when running deep learning workloads. It presents a kernel-centric optimization approach with a focus on mapping neural networks to a more efficient set of kernels (parallel functions executed on GPGPUs) that ensures better utilization. This involves optimizations from multiple levels: algorithm level aiming to leverage more hardware-friendly formulations, operator level to harness on-chip high bandwidth on GPGPUs, and kernel implementation level that maximizes the utilization of computational resources.