UCLA

The past decade has witnessed the fast growth and tremendous success of machine learning. However, recent studies showed that existing machine learning models are vulnerable to privacy attacks, such as membership inference attacks, and thus pose severe threats to personal privacy. Therefore, one of the major challenges in machine learning is to learn effectively from enormous amounts of sensitive data without giving up on privacy. This dissertation summarizes our contributions to the field of privacy-preserving machine learning, i.e., solving machine learning problems with strong privacy and utility guarantees.

In the first part of the dissertation, we consider the privacy-preserving sparse learning problem. More specifically, we establish a novel differentially private hard-thresholding method as well as a knowledge-transfer framework for solving the sparse learning problem. We show that our proposed methods are not only efficient but can also achieve improved privacy and utility guarantees.

In the second part of the dissertation, we propose novel efficient and effective algorithms for solving empirical risk minimization problems. To be more specific, our proposed algorithms can reduce the computational complexities and improve the utility guarantees for solving nonconvex optimization problems such as training deep neural networks.

In the last part of the dissertation, we study the privacy-preserving empirical risk minimization in the distributed setting. In such a setting, we propose a new privacy-preserving framework by combining the multi-party computation (MPC) protocol and differentially private mechanisms and show that our framework can achieve better privacy and utility guarantees compared with existing methods.

The methods and techniques proposed in this dissertation form a line of researches that deepens our understandings of the trade-off between privacy, utility and efficient in privacy-preserving machine learning, and could also help us develop more efficient and effective private learning algorithms.