Skip to main content
Open Access Publications from the University of California

UC Riverside

UC Riverside Electronic Theses and Dissertations bannerUC Riverside

Evasion Attacks on Network Intrusion Detection: Investigation, Automation, and Mitigation

  • Author(s): Wang, Zhongjie
  • Advisor(s): Qian, Zhiyun;
  • Krishnamurthy, Srikanth V.
  • et al.
No data is associated with this publication.
Creative Commons 'BY-ND' version 4.0 license

Stateful network protocols, such as the Transmission Control Protocol (TCP), play a significant role in the modern Internet, taking part in almost every network application running on billions of user devices, including computers, smartphones, IoT devices, vehicles, etc. However, due to inevitable ambiguities in network protocol specifications, discrepancies are prevalent among different network protocol implementations and even different versions of the same implementation. As a result, discrepancies could lead to severe security vulnerabilities. One kind of such vulnerabilities is caused by discrepancies between the network stack of a network intrusion detection system (NIDS) and those of the endhosts. A deliberate attacker could leverage the discrepancies to craft network traffic that will be interpreted differently by the NIDS and the endhosts, and then mount an attack that can bypass the NIDS. Furthermore, due to the statefulness of the network protocol, the attacker can manipulate the state on the NIDS to permanently disable the NIDS on any connection.Our research focuses on the study of discrepancies among TCP implementations of NIDSes and endhosts, towards understanding the exploitation of and defense against vulnerabilities caused by discrepancies. We start first by manually investigating the discrepancies and then move on to automated techniques. More specifically, 1) we first investigate the most powerful censorship firewall on the Internet and discover the discrepancies between its implementation and that of a Linux server, which allows an adversary to evade the firewall; 2) in order to automatically discover such implementation-level discrepancies, we develop a general approach which employs automated testing and symbolic execution techniques to automatically explore the program space of the Linux TCP stack and thereby discover network packets that can evade deep packet inspection used by modern stateful firewalls and intrusion detection systems; 3) we develop a systematic approach to extract comprehensive and high-fidelity models from various versions of the Linux TCP implementation and exhaustively discover all discrepancies between them; we then build a NIDS that incorporates the discovered discrepancies and is immune to all evasion attacks. Ultimately, we seek to develop widely used tools that employ automated testing techniques to significantly improve the effectiveness and efficiency in discovering discrepancies among stateful network protocol implementations and prevent attacks that exploit such discrepancies.

Main Content

This item is under embargo until July 20, 2023.