Preventing the Memory Errors in the Large-Scale C/C++ Software
Abstract
The C and C++ programming languages are highly valued for their flexibility inlow-level memory management and exceptional performance. They are widely used in various applications, including Linux Kernel, Google Chrome, Microsoft Windows, and Firefox. However, this emphasis on performance comes at the cost of memory safety, resulting in various memory errors, such as use-after-free, use-before-initialization (UBI) and type confusion, etc. These bugs not only impact system reliability but also pose significant security risks, potentially allowing attackers to gain control over the entire system. While the security impacts they bring are harmful, there lacks the efficient and effective approaches to detect them in the large scale software. In this dissertation, we would explains the challenges towards this topic and try to explore new approaches to tackle them. More specifically, we focus on two types of memory errors, use before initialization(UBI) and type confusion. In this dissertation, we (1) address that precise UBI analysis needs path-sensitive analysis and the current approaches either generate too many false positives or cannot scale to the large scale software. (2) studied the long presence of the bug once it merged into the Linux kernel, and proposed the incremental approaches called IncreLux to detect UBI bugs for new kernel commits. IncreLux is able to analyzing individual commit within minutes and thus avoid the buggy code merging into the kernel upstream. (3) analyzed some type confusion vulnerabilities and patches for Chome, as well as the popular sanitizers’ approach to mitigate type confusion bugs. One insight is that the developers encoded the type information to some fields of the data structure. Before the pointer casting, developers already added some type checks by looking at those fields. Therefore, by collecting such developers’ check, we could reduce sanitizer’s instrumentation if such type check is already performed. This aids in enhancing the efficiency of preventing type confusion bugs.