SC13 Home > SC13 Schedule > SC13 Presentation - Taming Parallel I/O Complexity with Auto-Tuning

SCHEDULE: NOV 16-22, 2013

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

Taming Parallel I/O Complexity with Auto-Tuning



TIME: 4:00PM - 4:30PM

SESSION CHAIR: André Brinkmann

AUTHOR(S):Babak Behzad, Huong Vu Thanh Luu, Joseph Huchette, Surendra Byna, Prabhat Mr., Ruth Aydt, Quincey Koziol, Marc Snir


We present an auto-tuning system for optimizing I/O performance of HDF5 applications and demonstrate its value across platforms, applications, and at scale. The system uses a genetic algorithm to search a large space of tunable parameters and to identify effective settings at all layers of the parallel I/O stack. The parameter settings are applied transparently by the auto-tuning system via dynamically intercepted HDF5 calls. To validate our auto-tuning system, we applied it to three I/O benchmarks that replicate the I/O activity of their respective applications. We tested the system with different weak-scaling configurations that generate 30 GB to 1 TB of data, and executed these configurations on diverse HPC platforms (Cray XE6, IBM BG/P, and Dell Cluster). In all cases, the auto-tuning framework identified tunable parameters that substantially improved write performance over default system settings. We consistently demonstrate I/O write speedups between 2x and 50x for test configurations.

Chair/Author Details:

André Brinkmann (Chair) - Johannes Gutenberg-University Mainz

Babak Behzad - University of Illinois at Urbana-Champaign

Huong Vu Thanh Luu - University of Illinois at Urbana-Champaign

Joseph Huchette - Rice University

Surendra Byna - Lawrence Berkeley National Laboratory

Prabhat Mr. - Lawrence Berkeley National Laboratory

Ruth Aydt - HDF Group

Quincey Koziol - HDF Group

Marc Snir - University of Illinois at Urbana-Champaign

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

The full paper can be found in the ACM Digital Library