UC Riverside

The operating system kernel security is critical for the overall system reliability, since the kernel runs in the high privileged mode and is often a part of the trusted computing base. The kernel vulnerabilities can cause severe consequences because they can be exploited by attackers to compromise many important security mechanisms (e.g., Linux's access control). In this dissertation, we try to identify and study various kernel security problems across its development cycle. More specifically, we (1) Identify some kernel design and implementation flaws by systematically analyzing an Android kernel memory management sub-system, namely ION. We discover, exploit, and develop a tool to help mitigate vulnerabilities related to these flaws. (2) Analyze a class of stealthy kernel vulnerabilities: the high-order taint style vulnerabilities. We then design and implement a novel automatic static program analysis to effectively and efficiently detect such vulnerabilities in the kernel testing phase. (3) Study the attacks against the deployed kernels by analyzing multiple representative Android one-click root apps. By reverse engineering, we extract and study hundreds of well crafted kernel root exploits from these apps and alarm the community of the security risks of abusing such apps. (4) Study a security weakness in the kernel maintenance phase: the delayed or missed security patch propagation. To help battle this problem we develop a tool to accurately test the security patch presence at the binary level, which can warn the defenders of the missed security updates. Our ultimate goal is to make the kernel more secure by analyzing and fixing security issues across its whole development cycle.