Supercomputing applications rely on strong scaling to achieve faster results on a larger number of processing units. But, at the strong-scaling limit, where communication is a relatively large portion of an application’s runtime, today’s state-of-the-art hybrid MPI+threads applications perform slower than their traditional MPI everywhere counterparts. This slowdown is primarily due to the supercomputing community’s outdated view: the network is a single device. NICs of modern interconnects feature multiple network hardware contexts. These parallel interfaces into the network are not utilized in MPI+threads applications today because MPI libraries still use conservative approaches to maintain MPI’s ordering constraints. MPI libraries do so because domain scientists today do not do a good job exposing logically parallel communication in their multithreaded MPI communication even though the existing MPI standard provides them with opportunities to do so. Only when domain scientists and MPI developers take a step forward together can we eliminate the communication bottleneck in MPI+threads applications.

This dissertation eliminates the communication bottleneck by bridging the two ends of the HPC stack—MPI library developers and domain experts—that typically do not talk to each other directly. Through collaborations with system researchers and MPI library developers, we develop a fast MPI+threads library capable of achieving scaling communication throughput similar to that of MPI everywhere and make high-speed multithreaded communication a reality. Through collaborations with domain scientists, we use various designs to expose logically parallel communication to the fast MPI+threads library on exemplar applications targeted to run on the upcoming exascale systems. Our conversations with the end-users—the domain experts—educate us on the usability aspects of the various designs. Hence, in addition to the performance comparisons of the various designs, we discuss the strengths and limitations of the different designs and provide our design recommendation for the supercomputing community. Through such collaborations on both ends of the HPC stack, we unlock the true potential of the MPI+threads programming model. Prominent modern applications and computational frameworks, such as Uintah, WOMBAT, and Legion, now perform significantly faster (up to 2x) at the strong-scaling limit.

Mozilla Research is developing Servo, a parallel web browser engine, to exploit the benets

of parallelism and concurrency in the web rendering pipeline. Parallelization results in

improved performance for pinterest.com but not for google.com. This is because the workload of a browser is dependent on the web page it is rendering. In many cases, the overhead of creating, deleting, and coordinating parallel work outweighs any of its benets. In this work, I model the relationship between web page primitives and a web browser's parallel

performance and energy usage using both regression and classication learning algorithms.

I propose a feature space that is representative of the parallelism available in a web page

and characterize it using seven key features. After training the models to minimize custom-de

ned loss functions, such a model can be used to predict the degree of parallelism available

in a web page and decide the optimal thread conguration to use to render a web page. Such

modeling is critical for improving the browser's performance and minimizing its energy usage.

As a case study, I evaluate the models on Servo's styling stage. Experiments on a quad-core

Intel Ivy Bridge (i7-3615QM) laptop show that we can improve performance and energy

usage by up to 94.52% and 46.32% respectively on the 535 web pages considered in this

study. Looking forward, we identify opportunities to tackle this problem with an online-learning

approach to realize a practical and portable adaptive parallel browser on various

performance- and energy-critical devices.