The International Conference for High Performance Computing, Networking, Storage and Analysis
Predictions of Large-scale QMCPack I/Os on Titan Using Skel.
Authors: Matthew Wezowicz (University of Delaware), Michael Matheny (University of Delaware), Stephen Herbein (University of Delaware), Jeremy Logan (Oak Ridge National Laboratory), Jeongnim Kim (Oak Ridge National Laboratory), Jaron Krogel (Oak Ridge National Laboratory), Scott Klasky (Oak Ridge National Laboratory), Michela Taufer (University of Delaware)
Abstract: Large-scale applications such as the Quantum Monte Carlo code QMCPack face a major challenge to preserve their scalability when fine-grained data are gathered and stored to disk or used for in-situ analysis.
Using an IO framework such as ADIOS allows us to address the trade-off by deploying different IO methods with little code modifications. Still the search for the most suitable methods and settings can be challenging. Ideally such a search can be conducted on a tool such as Skel that allows us to decouple the IO from the computation in real applications and to tune the IO on real supercomputers.
To allow tuning for applications with unbalanced generation of IO, we extended Skel to integrate the IO variability shown real world applications. We validated the Skel results against actual QMCPack IO performance data and showed that Skel results can be used to predict IO and steer choices for applications.