SC13 Home > SC13 Schedule > SC13 Presentation - Using Cross-Layer Adaptations for Dynamic Data Management in Large Scale Coupled Scientific Workflows

SCHEDULE: NOV 16-22, 2013

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

Using Cross-Layer Adaptations for Dynamic Data Management in Large Scale Coupled Scientific Workflows

SESSION: Optimizing Data Movement


TIME: 11:00AM - 11:30AM

SESSION CHAIR: Dhabaleswar K. (DK) Panda

AUTHOR(S):Tong Jin, Fan Zhang, Qian Sun, Hoang Bui, Manish Parashar, Hongfeng Yu, Scott Klasky, Norbert Podhorszki, Hasan Abbasi


As system scales and application complexity grow, managing and processing simulation data has become a significant challenge. While recent approaches based on data staging and in-situ/in-transit data processing are promising, dynamic data volumes and distributions,such as those occurring in AMR-based simulations, make the efficient use of these techniques challenging. In this paper we propose cross-layer adaptations that address these challenges and respond at runtime to dynamic data management requirements. Specifically we explore (1) adaptations of the spatial resolution at which the data is processed, (2) dynamic placement and scheduling of data processing kernels, and (3) dynamic allocation of in-transit resources. We also exploit coordinated approaches that dynamically combine these adaptations at the different layers. We evaluate the performance of our adaptive cross-layer management approach on the Intrepid IBM-BlueGene/P and Titan Cray-XK7 systems using Chombo-based AMR applications, and demonstrate its effectiveness in improving overall time-to-solution and increasing resource efficiency.

Chair/Author Details:

Dhabaleswar K. (DK) Panda (Chair) - Ohio State University

Tong Jin - Rutgers University

Fan Zhang - Rutgers University

Qian Sun - Rutgers University

Hoang Bui - Rutgers University

Manish Parashar - Rutgers University

Hongfeng Yu - University of Nebraska-Lincoln

Scott Klasky - Oak Ridge National Laboratory

Norbert Podhorszki - Oak Ridge National Laboratory

Hasan Abbasi - Oak Ridge National Laboratory

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

The full paper can be found in the ACM Digital Library