Deep learning has facilitated human-level performance on several tasks spanning a multitude of domains such as computer vision, natural language processing, medical analysis, gaming, retail, and marketing, just to name a few. The ability to solve a problem end-to-end, learn self-supervised high-level features from the data, and minimal hand-engineering have been key contributing factors in its success.

Due to the success of deep learning in related areas, it is also making in-roads into security. While a complete automation of a practical security system may be a remote prospect, we have seen many security sub-systems being upgraded with deep learning capabilities. For its self-learning capabilities, deep learning has been successfully used for enterprise-level network intrusion detection, malware detection and analysis, spam and phishing detection, and data privacy protection.

This work brings deep learning for security closer to the end-user. In addition to providing latency and scalability benefits, it enables a path away from privacy-invasive training and inference procedures.

In the first part of this work, I introduce Percival, an in-browser, deep learning powered native perceptual ad blocker implemented in two browsers. Percival advances the state of ad blocking and defends against a variety of attacks published against ad blockers, all while running purely client-side without any server intervention.

In the second part of this work, I present Boxer, a client-side Software Development Kit (SDK) and a server that can be used to prevent credit card fraud. Boxer’s client-side SDK scans user’s credit card and extracts high-level privacy-preserving features, which it then sends to the server for further processing. Boxer runs the entire deep learning inference client-side which ensures privacy-sensitive user data never leaves the user’s device.

While Percival and Boxer respect end-user privacy and run machine learning inference client-side, the performance differences in running these models on end-devices could result in the compromise of the system utility or introduce bias into the decision process. Percival could degrade the browsing experience and Boxer could unfairly block a user with a low-end device.

In the third part of this work, I quantify the performance differences of running Boxer on the breadth of the devices one can see in distribution. Specifically, I perform a large-scale in-field study of running Boxer on front-end devices and quantify the impact of hardware diversity on the performance and reliability of Boxer’s machine learning pipeline. I identify the key performance metrics and design strategies that are critical for any on-device machine learning application.

Finally, in the last part of this work, I present a new anti-fraud payment card scanning system called Daredevil. Daredevil incorporates insights from the in-field measurement study and works well across the broad range of performance characteristics and hardware configurations found on modern mobile devices.