UC San Diego

Traditional network resource allocation is not scalable because it requires per-flow state, large amount of memory in switches and routers, and control overhead. In this dissertation, we propose innovative and scalable mechanisms for network traffic management in three emerging contexts: network event loggers, network load balancing, and cloud services in data centers. First, we describe a probabilistic event logger called Carousel to collect unique items in a large stream of online events. By theoretical analysis, we prove that Carousel can collect almost all items with high probability. Our simulation and implementation prototype show an improvement factor of ten in event collection time. Second, we design a new load balancing algorithm called Flame that is implementable in high speed switches with small memory usage. Flame achieves fine granularity of load balancing at sub-flow level and binds flows to hash functions. Through trace simulation, we show that Flame can improve our load balancing performance metrics by an order of magnitude. Furthermore, Flame allows graceful degradation to the standard ECMP in the worst case. Lastly, we propose a mechanism called NetShare to provide predictable network resource allocation for cloud services based on simple administrative weights. We describe mechanisms to implement and scale NetShare to a large number of services using a generalization of Stochastic Fair Queueing. We validate our NetShare design on a hardware testbed with MapReduce workloads