UC San Diego

Resource disaggregation proposes a next-generation architecture for data center resources.System components like compute, memory, storage, and accelerators are separated from one another by a fast network and composed dynamically into virtual servers when required. This paradigm promises dramatically improved resource utilization, scalability, and flexibility, but introduces dramatic challenges in terms of performance and fault tolerance. Memory is among the most difficult resources to disaggregate. CPUs currently expect DRAM to ultra low latency, high bandwidth, and to share it’s failure domain. In particular increased latency from network round trips dramatically shifts the performance of existing shared data structures designed for local DRAM. In this thesis I explore the challenges of sharing disaggregated memory with hundreds of CPU cores. First in SwordBox I present a system which utilizes a centralized programmable switch to cache data structure state and dramatically improve key-value workload performance. Second I present a new key-value store RCuckoo which is designed to leverage RDMA and reduce round trips when accessed by CPU’s over a network. Both systems demonstrate significant performance improvements over the existing state of the art.