Collaborative Research: CIRC: New: Next Generation Message-Passing Parallel Programming for Heterogeneous Architectures

About This Grant

Each generation of emerging supercomputers advances available compute power, enabling new scientific discoveries. However, parallel solvers and simulations often achieve only a small fraction of peak performance of the underlying supercomputer system because of inter-process communication overheads and complexities. In these applications, communication is handled by the Message Passing Interface (MPI) application programmer interface, provided through-standards-based implementations (via libraries and header files). The achieved performance associated with the numerous possible types of inter-process communication vary greatly, depending not only on the underlying architecture but also the version of MPI that has been hand-tuned for the given system. Typically, a proprietary vendor-dependent implementation is available on the largest systems, with open-source versions available on cluster systems. While multiple open-source versions of MPI exist, installing an appropriately tuned version requires significant expertise in both the MPI framework and the hardware of the system. The gap between both the performance and modern feature coverage of production MPI's on large-scale systems can present challenges for application developers seeking to reach higher achievable performance and manage the complexities of modern architectures. This project will therefore develop an open-source cyberinfrastructure, MPI Advance, which utilizes low-level communication routines, such as send and receive methods, from the hand-tuned system MPI installation to publish new and improved application programmer interfaces (APIs) to drive higher achievable performance without the need to supplant the built-in MPI implementations on large-scale systems. MPI Advance uses the MPI-eXtension framework to provide optimizations within existing MPI APIs, extensions to these standard APIs, and entirely new routines that do not conform to the current standard but can lead to significant performance optimizations within existing parallel applications. This cyberinfrastructure will be integrated within widely used solver frameworks, such as PETSc, hypre, and Trilinos, enabling straightforward integration into any application relying on these code bases. Further, MPI Advance will provide a framework for research within MPI, allowing researchers to test new extensions across a range of applications and achieve new community best practices before these new features and APIs are proposed adoption within future editions of the MPI standard. MPI Advance will be made broadly available under BSD-3 license and be published via GitHub, in addition to documentation and supporting infrastructure on web pages dedicated to its user base. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.