UC Riverside

Detecting intrusions is usually the first step in containing a security breach. In this dissertation, we focus on how we can understand and profile IoT malware behavior in order to find intrusions. Specifically, we want to develop a holistic view of how the malware behaves once installed in an IoT device, which will encompass: (a) how does the malware communicate with its Command and Control server, (b) how does the malware communication and infrastructure change over time, and (c) how does a network-affected data flows through the device (which is known as taint analysis).

In the first chapter of this thesis, we propose a novel solution for finding IP and port addresses of live Command and Control (CnC) servers of IoT botnets. Furthermore, we present a solution for detecting intrusions using only network level patterns given a malware binary. Our novelty lies in activating the malware binary and extracting patterns that can precisely indicate intrusions without the need of deep packet inspection.

In the second chapter, we conduct an extensive study of the behaviors of both bots and CnC servers over time. In more detail, we probe IP networks that show signs of CnC hosting and search for CnC servers among neighbor hosts on a daily basis. This approach allows us to find CnC servers in an active manner; we find CnC servers even before their corresponding malware samples are collected by honeypots. In addition, the collected data provides insight on the network infrastructure of IoT botnets. Finally, we shed light on the non-CnC traffic of the IoT malware that mainly tries to spread the malware.

In the third chapter, we focus on taint analysis where the goal is to understand the flow of data within a system. In the Intrusion Detection context, the input data is the network traffic e.g. a command from the CnC. We discuss how dynamic taint analysis can be selectively applied for whole system analysis in intrusion detection systems, and show how this approach can speed up the system.

Overall, this thesis provides a fundamental step in understanding IoT malware behavior, which can lead to better defenses in terms of detecting and containing the associated damage.