UC San Diego

Hardware accelerators are becoming increasingly important in (1) meeting application performance demands and (2) mitigating datacenter software overheads. However, offloading computation to accelerators incurs fundamental overheads, which must be accounted for when measuring their benefits. In this thesis, we study the characteristics of a modern compression accelerator and show that only certain offload granularities yield performance gains. Further, we show that compressing large buffers (≥ 8KB) with an off-chip accelerator yields latencies within 25% of the offload latency for an integrated compression accelerator that resides on the same chip as a SmartNIC’s ARM cores, indicating the viability of off-chip acceleration. We use these insights to design and implement a microsecond-scale operating system scheduler that selectively offloads the parts of an application that benefit from hardware acceleration. Moreover, we design scheduling policies that decide whether hardware offload should be performed synchronously or asynchronously, depending on workload characteristics. Preliminary evaluation on synthetic applications closely modeled after real datacenter workloads shows that our system achieves up to 28% lower median latency and up to 3.8x higher overall throughput than a software-based approach by efficiently offloading compression to hardware accelerators.