The SpiceC Parallel Programming System
- Author(s): Feng, Min
- Advisor(s): Gupta, Rajiv
- et al.
As parallel systems become ubiquitous, exploiting parallelism becomes crucial
for improving application performance. However, the complexities of developing
parallel software are major challenges. Shared memory parallel programming
models, such as OpenMP and Thread Building Blocks (TBBs), offer a single view
of the memory thereby making parallel programming easier. However, they
support limited forms of parallelism. Distributed memory programming models,
such as the Message Passing Interface (MPI), support more parallelism types;
however, their low level interfaces require great deal of programming effort.
This dissertation presents the SpiceC system that simplifies the task of
parallel programming while supporting different forms of parallelism and
parallel computing platforms. SpiceC provides easy to use directives to
express different forms of parallelism, including DOALL, DOACROSS, and
pipelining parallelism. SpiceC is based upon an intuitive computation model
in which each thread performs its computation in isolation from other
threads using its private space and communicates with other threads
via the shared space. Since, all data transfers between shared and
private spaces are explicit, SpiceC naturally supports both shared and
distributed memory platforms with ease.
SpiceC is designed to handle the complexities of real world applications.
The effectiveness of SpiceC is demonstrated both in terms of delivered
performance and the ease of parallelization for applications with the
Applications that cannot be statically parallelized due to presence of
dependences, often contain large amounts of input dependent and dynamic
data level parallelism. SpiceC supports speculative parallelization
for exploiting dynamic parallelism with minimal programming effort.
Applications that operate on large data sets often make extensive use of
pointer-based dynamic data structures. SpiceC provides support for
partitioning dynamic data structures across threads and then distributing
the computation among the threads in a partition sensitive fashion.
Finally, due to large input sizes, many applications repeatedly perform
I/O operations that are interspersed with the computation. While traditional
approach is to execute loops contain I/O operations serially, SpiceC
introduces support for parallelizing computations in the presence
of I/O operations.
Finally, this dissertation demonstrates that SpiceC can handle the challenges
posed by the memory architectures of modern parallel computing platforms.
The memory architecture impacts the manner in which data transfers between
private and shared spaces are implemented. SpiceC does not place the
the burden of data transfers on the programmer. Therefore portability of SpiceC
to different platforms is achieved by simply modifying the handling of
data transfers by the SpiceC compiler and runtime. First, it is
shown how SpiceC can be targeted to shared memory architectures both with and
without hardware support for cache coherence. Next it is shown how accelerators
such as GPUs present in heterogeneous systems are exploited by SpiceC. Finally,
the ability of SpiceC to exploit the scalability of a distributed-memory system,
consisting of a cluster of multicore machines, is demonstrated.