Lawrence Berkeley National Laboratory

As we enter the era of petascale computing, system architects must plan for machines composed of tens of thousands or even hundreds of thousands of processors. Although fully connected networks such as fat-tree interconnects currently dominate HPCnetwork designs, such approaches are inadequate for thousands of processors due to the superlinear growth of component costs. Traditional low-degree interconnect topologies, such as the 3D torus, have reemerged as a competitive solution because the number of switch components scales linearly with the node count, butsuch networks are poorly suited for the requirements of many scientific applications. We present our latest work on a hybrid switch architecture called HFAST that uses circuit switches to dynamically reconfigure a lower-degree interconnect to suit the topological requirements of each scientific application. This paper expands upon our prior work on the requirements of non-adaptive applications by analyzing the communication characteristics of dynamically adapting AMR code and presents a methodology that captures the evolving communication requirements. We also present a new optimization that computes the under-utilization of fat-tree interconnects for a given communication topology, showing the potential of constructing a "fit-tree" for the application by using the HFAST circuit switches to provision an optimal interconnect topology for each application. Finally, we apply our new optimization technique to the communication requirements of the AMR code to demonstrate the potential of using dynamic reconfiguration of the HFAST interconnect between the communication intensive phases of a dynamically adapting application.