HPCUG Presentation

Comparing Parallel Programming Paradigms And Their Applicability

Henry Strauss, Hewlett-Packard GmbH

Presenting Author: Strauss (strauss@rsn.hp.com)

Abstract

We present an overview of different ways to exploit parallism inherent to various application areas of high-performance computing like mechanical design automation (MDA), electronic design automation (EDA), and computational chemistry.
From a computer design point of view, one typically distinguishes between shared- and distributed-memory programming since these are the native ways to program machines with global and local memory, respectively.
However, some interesting developments happened over the past few years:
1. The native programming method of distributed-memory machines, message passing, became available on shared-memory systems, too, because the Message Passing Interface (MPI) has been established as the de-facto standard.
2. Efficiently implemented MPI libraries on global-memory machines give better performance, in terms of latency and bandwidth numbers as well as of moderately parallel application runtimes, than their counterparts on local-memory machines.
3. Although pure message passing seems to scale better, i.e. to larger numbers of cpus, its applicability and efficiency is limited to certain classes of problems with certain characteristics like static load balancing.
Distributed-memory programming via message-passing often, for example in computational fluid dynamics (CFD), means domain decomposition. Although this is a viable approach some limitations with it on distributed-memory machines begin to become visible:
E.g. local and/or adaptive mesh refinement is a feature of state-of-the-art MDA software packages which is hard to implement on a distributed-memory machine using message-passing as the only paradigm available on such systems. And when its implemented the parallel efficiency drops dramatically since it is too expensive to redistribute the mesh data on a local-memory machine.
So, it is necessary to combine different programming approaches in order to achieve new levels of functionality and performance.