Lawrence Berkeley National Laboratory

In the exascale era, comprehensive energy accounting is critical for sustainable HPC. Standard monitoring captures CPU and GPU power, but the energy footprint of interconnects, including switches, NICs, and PCIe/NVLinks, remains hidden due to limited hardware sensors. We address this with a software-centric methodology using targeted microbenchmarks on the Perlmutter (HPE Cray EX) system at NERSC. By correlating controlled stress on specific network components with job- and rack-level power telemetry, we isolate each component’s energy consumption, quantifying dynamic energy per bit (pJ/bit) for active communication while separating constant-power overhead. Integrating network bandwidth measurements from vendor tools such as NVIDIA DCGM and CrayPat enables fine-grained, application-level estimates of network energy use, unattainable with standard monitoring. This approach establishes a generalizable framework for evaluating network energy, providing actionable insights for designing and operating power-efficient, sustainable supercomputing systems.