Peer loss [1] is a new family of loss functions proposed to deal with the problem of learning with noisy labels. It claims to handle a wide range of label noise in binary classification tasks without explicitly estimating the noise rates. Numerical experiments demonstrate the effectiveness of peer loss. However, its extension to the multi-class classification remains unclear, and its working mechanism is not fully understood.

In this thesis, we study the theory of peer loss from three distinct perspectives. Follow the original method in [1], we first consider the multi-class extension of peer loss and investigate its noise tolerance properties. From this perspective, we see peer loss as a class of loss functions inspired by the truthful and proper scoring rules in the peer prediction literature. It turns out that this perspective is a static one and cannot provide a satisfactory explanation of how peer loss works in practical training. To gain an intuitive picture of the working mechanism, we further develop a divergence perspective towards peer loss, expressing it as the difference between two KL divergences. Thus, we recognize that peer loss has a built-in regularization effect, encouraging the model to make confident predictions. This regularization effect partially explains why peer loss works well under the label noise, as the existence of noise often blurs the data distribution and makes the resulting model prediction uncertain. Finally, we show that peer loss potentially suggests a new type of risk in decision theory, i.e., the correlation risk. This new perspective helps us to understand better what the model learns when trained with peer loss. To complete the discussion of the correlation risk perspective, we develop a novel method to investigate the training dynamics of peer loss. This dynamical analysis justifies that with peer loss, the resulting model tends to grasp the positive correlations in the training datasets.

In addition to the theoretical analysis, we also carry out extensive numerical experiments. Those experiments on benchmark image datasets demonstrate the effectiveness of peer loss on the multi-class classification tasks under a wide range of label noise. Our experiments on the 2-dimension synthetic dataset reveal that the models trained with peer loss tend to produce hard decision boundaries. This phenomenon accords with our theoretical analysis that peer loss encourages the model to make confident predictions.

Text classification is one of the most fundamental tasks in Natural Language Processing. How to effectually utilize the unlabeled dataset in text classification and apply weakly supervised learning methods to further improve the performance based on the existing labeled dataset, especially for supervision-starved tasks (hard to obtain high-quality labeled data), is challenging. In this PhD thesis, we show several studies of weakly supervised learning methods in text classification.

We first focus on improving the accuracy and interpretability in text classification tasks using weakly supervised learning methods with the help of unlabeled dataset. More specifically, we proposed several new methods to further improve the accuracy and interpretability on both of two main research directions in weakly supervised learning methods: learning with noisy labels and semi-supervised learning. For learning with noisy labels, we proposed two weakly supervised learning aided methods on the special supervision-starved text classification task: Research Replication Prediction. For semi-supervised learning, we presented a new weakly interpretable model to improve the interpretability on the long text classification tasks. We also proposed a new ensemble method to assign better pseudo or noisy labels to the samples in the unlabeled dataset for semi-supervise learning methods.

Furthermore, we conducted the research on fairness on weakly supervised learning. More specifically, we reveal the disparate impacts in different sub-populations (e.g., race and gender) when applying the semi-supervised learning methods. Finally, we also contribute a weakly supervised learning benchmark (Research Replication Prediction) to the community.

Since 1960, researchers have used proxies, such as debt-to-income ratio, loan-to-value ratio, combined-loan-to-value ratio, and the credit score of borrowers to measure mortgage loan performance. These variables provide a static view of mortgage defaults. However, mortgage defaults must be examined in a dynamic framework. Therefore, I used macroeconomic variables, such as real gross domestic product, the consumer price index, real median household income, interest rates, and the national home price index combined with the static variables to measure the relationships of all these variables with the default rate. I find that debt-to-income ratio, loan-to-value ratio, national home price index, and unemployment rate positively associated with the default rate, while the real gross domestic product and the real median household income negatively associated with default rate. I also find the real median household income is the most critical macroeconomic factor in predicting the default rate.

Heterogeneous computing has seen a great rise in the age of big data. In particular, heterogeneous computing systems with GPUs are able to deliver exceptional performance with better energy efficiency, thus equip us with great power to deal with the enormous yet fast growing volume of data.

In order to exploit the power of such heterogeneous systems with GPUs, we have to address the problems from three closely related aspects. First, we need to design and implement algorithms best suited to the CPU and the GPU, and schedule workloads to the best processors. Second, we have to maximize the utilization of the system resources for better performance and higher power efficiency. Third, we should provide efficient I/O management and data transfer mechanisms to accommodate the increasing amount of data.

This dissertation explores the systems and algorithm support for efficient heterogeneous computing with GPUs to respond to the above three problems. We first present a system called Hippogriff, which enables efficient direct data transfers between the SSD and the GPU. Hippogriff is able to dynamically choose the best data transfer route for the GPU to improve the overall system performance and efficiency.

We then focus on improving the resource utilization in the context of MapReduce, and present SPMario, a system to scale up MapReduce with the GPU via optimized I/O handling and task scheduling. SPMario proposes I/O Oriented Scheduling to coordinate concurrent task execution in a way that minimizes the idle time of the system resources while avoiding I/O contention.

Last, we present a heterogeneous search engine called Griffin, which explores new parallel algorithms and task scheduling to meet the rigorous requirement of query latency in Web search. Griffin uses a dynamic intra-query scheduling algorithm to break a query into sub-operations, and adaptively schedules them to the state-of-the-art CPU search implementation and to our new GPU-based search kernels, based on the ever-changing runtime characteristics of the queries.

Overall, we demonstrate our systems address the above three problems, and can improve the performance and efficiency of the heterogeneous computing systems with GPUs.

In living organisms, numerous biochemical reactions occur synergistically, allowing the organisms to grow and reproduce, convert food to energy, maintain the structure, and response to the environment – all activities that we called “life”. As one of the most important types of molecules for any living organism, protein controls and mediates all these biological processes; malfunction of protein often induces the body to an abnormal state namely disease. Viewed from the perspective of disease mechanisms, protein therapy, which treats diseases by delivering therapeutic proteins directly to redress disorders, represents a tremendous opportunity to alleviate many incurable diseases. However, intrinsic problems, including low stability, low permeability, and high immunogenicity, hamper the clinic application of proteins. Development of advanced protein delivery systems, which can properly address these issues, has been considered as a practical solution that could enable the therapeutic use of a wide scope of proteins in disease treatments.

In this dissertation, novel delivery systems have been developed based on the protein nanocapsule technology, providing several practical methods to overcome essential issues that hinder the application of therapeutic proteins, but have not yet been properly resolved by current protein delivery systems. These common issues include the efficiency of protein intracellular delivery, co-delivery of multiple proteins, and the plasma half-life and immunogenicity of proteins when delivering systemically. Based on these issues, the dissertation research can be outlined briefly with the following three topics:

1. Intracellular delivery of any protein(s) with optimal efficiency. This part of work demonstrated a polymer-based nano-carrier that was constructed by utilizing multiple weak interactions, which offers the feasibility of loading and delivering proteins without any chemical modification, as well as finely tuning the surface properties of the nano-carriers to achieve optimal delivery efficiency. The intracellular delivery capability of this nano-carrier was first demonstrated by delivering bioactive transcription factors into cells (Chapter 3). Combining with single-protein nanocapsule and microfluidic platform, any protein can be intracellularly delivered using this nano-carrier and the delivery efficiency can be optimized with a high-throughput systhesis and screening system (Chapter 5), providing a fundamental platform for researches on stem cell reprogram and regenerative medicine.

2. Co-delivery of multiple proteins that function synergistically. In eukaryotic cells, most biological processes are accomplished by multiple enzymes or proteins, which are usually spatially confined with precise ratio control in order to function correctly and optimally. Demonstrated in this part of work (Chapter 1), defined types and amount of enzymes were assembled and co-encapsulated into enzyme nanocomplex, which ensures those enzyme functioning effectively no mater being delivered into local tissues or diluted system like blood stream. This strategy provides a feasible method to construct protein-based multi-functional nanostructure as needed without engineering protein sequence. From the perspective of developing protein therapeutics, this system offers a practical method to detoxify the by-product of many enzymes, enabling safe use of them for therapeutic purposes.

3. Prolong circulation time and reduce immunogenicity of therapeutic proteins. One major problem of most therapeutic proteins is the fast clearance by immune system, which significantly reduces or completely diminish their therapeutic effects. In this part (Chapter 3), we designed a general method that can prolong the plasma half-life of most proteins significantly by minimizing their interactions with serum proteins, blood cells, tissues and organs with a protein-adsorption-resistant polymer shell. With this method, the overall therapeutic effect was enhanced after systemic administration, which was demonstrated with uricase as model therapeutic protein. This strategy also inhibits immune responses against exogeneous proteins, which can broaden the scope of proteins with therapeutic potentials.

Overall, this dissertation research established various methods to endow new surface properties to proteins and enzymes as needed. Based on these work, one can expect that increasing number of protein therapeutics will be developed and widely applied for curing diseases in the near future.

Reactive oxygen species (ROS) are a class of reactive molecules that are continuously generated as a byproduct of aerobic metabolism in animals and plants. ROS were reported to play important roles in both signal transduction and homeostasis. In normal conditions, cells could maintain a delicate balance between oxidants and antioxidants. However, when ROS level go beyond its normal range, various negative effects could be induced including cytotoxicity, mutagenicity and carcinogenicity. Developing ROS detection tools and methods are of pivotal importance here to help further ROS targeted studies.

Heavy metals are another class of molecules that occur naturally with a high atomic weight. Some of them are essential nutrients that are necessary for different biological and physiological functions. Nevertheless, concerns on their health effects were raised along with their increasing usage in industry, agriculture and modern technology. Exposure to heavy metals were reported to cause carcinogen, organ damage and genotoxicity.

Regarding the molecule level mechanism of above toxicants, proteins were reported to be one of the major targets. Protein covalent modifications occur frequently in all kinds of proteins and one of the most important class is the histone protein. Post-transitional modifications (PTM) to histone, including acetylation, methylation, phosphorylation, ubiquitination etc. Other than modification types, histone modification studies are further complexed by possibilities of different modification sites. In this dissertation, Chapter 1 is an introduction on ROS, heavy metal and PTMs, as well as current analytical techniques. Chapter 2 describes a novel microgel based amplification technique aiming to study ROS. Glucose and cholesterol are also measured based on the sample technique. Chapter 3 is reporting a deep cavitand based sensing method used for histone methylation study. Chapter 4 and 5 are more in-depth studies on sensing of different PTMs with deep cavitand based sensor array. Chapter 6 describes a further application of cavitand sensor array in metal detection. In Chapter 7, future directions and some preliminary studies are described.

Recent work has connected adversarial attack methods and algorithmic recourse methods: both seek minimal changes to an input instance which alter a model’s classification decision. It has been shown that traditional adversarial training, which seeks to minimize a classifier’s susceptibility to malicious perturbations, increases the cost of generated recourse; with larger attack tolerances (known as attack radii) during adversarial training correlating with higher recourse costs. From the perspective of algorithmic recourse, however, the appropriate adversarial training radius has always been unknown. Another recent line of work has motivated adversarial training with adaptive adversarial training radii to address the issue of instance-wise variable adversarial vulnerability, showing success in domains with unknown attack radii. This work studies the effects of adaptive adversarial training on algorithmic recourse costs, establishing that the improvements in model robustness induced by adaptive adversarial training show minimal effects on algorithmic recourse costs. This provides a potential avenue for affordable robustness in domains where recoursability is critical.

In the context of artificial intelligence (AI) or machine learning (ML), we speak of the "alignment" of an AI system's behavior with human goals, values, and ethical principles. "The alignment problem" has proven challenging, and as the capabilities and applications of AI rapidly advance, the shortcomings of standard solutions are increasingly consequential. This dissertation focuses on an often overlooked but critically important complication to the alignment problem: Socially-consequential AI systems affect their environment (involving, for example, human populations) and are therefore subject to dynamical feedback driven by other agents. We address three central questions: (1) As intelligent agents adapt to each other, does a system aligned using current leading approaches remain aligned? (2) Can we anticipate and utilize adaptive agents' reactions to data-driven policy to achieve aligned objectives dynamically? (3) How can we guarantee alignment for AI systems that interact with complex, multiagent environments that are difficult to model or predict? We will address these questions using the theoretical framework and experimental tools of machine learning---integrating concepts from dynamical systems, evolutionary game theory, constrained optimization, and control theory. We hope to demonstrate that a dynamical systems approach to deployed AI is not only necessary but beneficial to the goal of alignment.