Understanding Internet traffic is important in a number of different scenarios. Internet service providers (ISPs) would like to understand traffic demand at different points in the network while taking capacity planning measures for the future. Router manufacturers would benefit from understanding traffic characteristics while performing queue managements studies. Similarly, existing Internet traffic interacts with and influences the performance of any new deployed service that we wish to test on the Internet. In order to assist with these scenarios, we propose Internet-like traffic generation for local testbeds with the ability to project traffic demands into alternate scenarios. Achieving this goals requires addressing challenges. For instance, traffic mix changes over time and application semantics keep evolving. Moreover, Internet traffic exhibits a statistically rich structure at multiple timescales. The cornerstone of this thesis is the contribution of a tool called Swing which addresses the above challenges. It is the first realistic and responsive traffic generator that can reproduce observed burstiness properties in Internet traffic at multiple time scales. We validate Swing using publicly available traffic traces and then present two of the many different uses it enables. We demonstrate that it is important to consider background traffic for evaluation of distributed systems. The fact that Internet traffic is bursty means that it interacts with application traffic in complex ways, and the outcome cannot be predicted a- priori. We propose that system evaluation should proceed by subjecting the application to realistic traffic settings, and more importantly, to a variety of different scenarios. In particular, we demonstrate that some of the common techniques the community has been using, including Poisson and CBR (constant bit rate) traffic, can lead to misleading conclusions. Finally, we demonstrate the utility of Swing to project traffic demands into the future. We start by exposing the limitations of existing techniques due to their inability to understand/model evolution. Thus, we propose a simple mechanism that network administrators could use to generate traffic demands based on intuitively projected what-if scenarios

Large-scale network services can consist of tens of thousands of machines running thousands of unique software configurations spread across hundreds of physical networks. Testing such services for complex performance problems, configuration errors, and fault tolerance remains a difficult problem. Existing testing techniques, for example through simulation or running smaller instances of a service, have limitations in predicting overall service behavior. Although technically and economically infeasible at this time, testing should ideally be performed at the same scale and with the same configuration as the deployed service. We present DieCast, an approach to scaling network services; we multiplex all of the nodes in a given service configuration as virtual machines (VM) spread across a much smaller number of physical machines in a test harness. CPU, network, and disk are then accurately scaled to provide the illusion that each VM matches a machine from the original service in terms of both available computing resources and communication behavior to remote service nodes. We present the architecture and evaluation of a system to support such experimentation and discuss its limitations. We show that for a variety of services, including a high-performance, cluster-based file system, and resource utilization levels, DieCast matches the behavior of the original service while using a fraction of the physical resources.

Pre-2018 CSE ID: CS2007-0910