UC Riverside

Energy efficiency and sustainability of data centers have become more challenging as cloud services evolve and become more computation-heavy, particularly with machine learning (ML) training and inference. The growing demand for ML-intensive workloads has led to the integration of more powerful hardware, such as Graphics Processing Units (GPUs), which introduces new sustainability and energy-efficiency challenges for data centers providing ML cloud services.

In the first research direction, we address energy-efficiency challenges in data centers offering machine learning inference cloud solutions. Addressing these inefficiencies requires a comprehensive characterization of the system. To this end, we propose GPU-NEST, a characterization framework targeting the energy efficiency of multi-GPU inference servers. To further implement energy efficiency techniques into inference servers, we developed ScaleServe, a scalable multi-GPU inference server capable of serving inference requests for multiple models concurrently. Additionally, we introduce WattWiser, an inference request scheduling policy that enables the sharing of GPUs among models during inference while maintaining Quality of Service (QoS), leading to efficient GPU resource utilization and lower power consumption. Collectively, these solutions resulted in a significant reduction in power consumption and improved GPU resource utilization.

In the second research direction, we focus on the sustainability challenges facing machine learning data centers. As these centers are predicted to consume significant amounts of electricity, they present a barrier to achieving net-zero carbon goals. We developed a framework, PowerMorph, that enables data centers to participate in frequency regulation programs. By engaging in these regulation services, data centers can help stabilize power grids that increasingly rely on intermittent renewable energy sources like wind and solar. This participation reduces dependence on fossil fuel-based power plants for grid balancing, thus lowering overall carbon emissions and promoting the integration of renewable energy. In EcoCenter, we target GPU-accelerated data centers due to higher power consumption and dynamic power range, and introduce the concept of exogenous carbon, which quantifies the reduction in carbon footprint that data centers can achieve by providing frequency regulation services (RS) to the power grid. Providing these services allows power grids to utilize more renewable resources in the long term, leading to overall power grid carbon reduction and increased sustainability.