Lawrence Berkeley National Laboratory

In solving Partial Differential Equations, such as the Barotropic equations in ocean models, on Distributed Memory Computers, finite difference methods are commonly used. Most often, processor subdomain boundaries must be updated at each time step. This boundary update process involves many messages of small sizes, therefore large communication overhead. Here we propose a new approach which expands the ghost cell layers and thus updates boundaries much less frequently ---reducing total message volume and grouping small messages into bigger ones. Together with a technique for eliminating diagonal communications, the method speedup communication substantially, up to 170 percent. We explain the method and implementation in details, provide systematic timing results and performance analysis on Cray T3E and IBM SP.