UC Santa Barbara

In today’s high-bandwidth culture, network managements face a set of challenges to those of previous decades. Traditional approaches rely on the static and manually configured rules, which are proposed by the network designers through years of study. The rules fail to handle the dynamic traffic as well as the increasingly heterogeneous operating conditions that come with the modern network environment. Besides, the traditional network design paradigm is no longer effective as human understanding of the complicated network becomes much harder and costly.

Recently, data-driven paradigm has been proposed to fill gaps in human understand- ing of complicated tasks. Data-driven paradigm allows the solutions to be learnt directly from the data produced in the task. Thanks to the rapid development of network monitoring tools over the years, we are able to obtain a large amount of network measurement data that makes it feasible to apply data-driven paradigm in network management.

This dissertation aims at designing and managing the networked systems based on data-driven paradigm. However, when applying data-driven approaches into the real-world tasks, there are multiple challenges. The key contribution of this dissertation is to provide solutions to address three fundamental challenges: First, the measurement data may be unreliable due to factors of measurement bias, hardware, environment and human artifacts. The bad quality of the data will affect solution learning. Second, regardless of the data issues coming from the measurement, the real-world network data itself can be complicated (e.g., bias) which requires specially designed learning algorithms to handle. Third, we need to improve the scalability of solution (model) for better efficiency. In this dissertation, we propose practical data-driven designs for three real-world network tasks: Radio-Frequency (RF) transmitter localization, network anomaly detection and spectrum anomaly detection.

First, we improve the performance of RF transmitter localization by filtering out unreliable measurement data. We propose a novel application of unsupervised learning to detect hidden correlation between measurement instances, their features, and localization accuracy. We use the key features to identify the types of measurement data that correlate well with high or low prediction accuracy. By only using the measurements of high prediction accuracy, we are be able to improve the localization accuracy.

Second, we predict network issues for Network Function Virtualization (NFV). NFV is a new network architecture, and there’s less domain knowledge of NFV management. We propose to learn the prediction rules from the system logs via deep learning. Since network issues are rare, we utilize Long Short-Term Memory (LSTM) to detect anomalies from the data that can potentially be used as early indicator of network issues that would typically result in trouble tickets.

Finally, we explore the design of a general, scalable system for detecting spectrum anomalies in wide-area LTE networks. We address the challenge by building context- agnostic models for spectrum usage and applying transfer learning to minimize training time and dataset constraints. The end result is a practical DNN model that can be easily deployed on both mobile and static observers, enabling timely detection of spectrum anomalies across LTE networks.

In summary, we propose a suite of algorithms and solutions driven by domain-specific insights to make data-driven designs practical and high-performing. We hope our studies could provide insights for researchers to explore new data-driven paradigms for future networking research.