The International Conference for High Performance Computing, Networking, Storage and Analysis
PGAS Models Using an MPI Runtime: Design Alternatives and Performance Evaluation.
Authors: Jeffrey Daily (Pacific Northwest National Laboratory), Abhinav Vishnu (Pacific Northwest National Laboratory), Bruce Palmer (Pacific Northwest National Laboratory), Hubertus van Dam (Pacific Northwest National Laboratory)
Abstract: In this paper, we explore the suitability of using MPI in designing a scalable PGAS communication subsystem. We focus on the RMA communication in PGAS models which typically includes get, put, and atomic memory operations. We perform an in-depth exploration of design alternatives based on MPI including using a semantically-matching interface such as MPI-RMA, as well as not-so-intuitive MPI interfaces such as two-sided, multi-threading, and dynamic process management. We propose a novel design which leverages a combination of highly-tuned MPI two-sided semantics and an automatic, user-transparent split of MPI communicators to provide asynchronous progress. We implement the asynchronous progress ranks (PR)-based approach and other approaches within ComEx - Communication runtime for Exascale, a communication subsystem for Global Arrays. Our performance evaluation includes numerous benchmarks as well as a computational chemistry application. Our PR-based approach achieves a 2.17x speed-up on 1008 processors over the other MPI-based designs.