SCHEDULE: NOV 16-22, 2013

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

SDQuery DSI: Integrating Data Management Support with a Wide Area Data Transfer Protocol

SESSION: Data Management in the Cloud

EVENT TYPE: Papers

TIME: 11:00AM - 11:30AM

SESSION CHAIR: Erwin Laure

AUTHOR(S):Yu Su, Yi Wang, Gagan Agrawal, Rajkumar Kettimuthu

ROOM:205/207

ABSTRACT:
In many science areas where datasets need to be transferred or shared, rapid growth in dataset size, coupled with lack of increase in wide area data transfer bandwidth, is making it extremely hard for scientists to analyze the data. This paper addresses the current limitations by developing SDQuery DSI, a GridFTP plug-in which supports flexible server-side data subsetting over HDF5 and NetCDF data formats. The GridFTP server is able to dynamically load this tool to download any data subset. Different queries types (query over dimensions, coordinates and values) are supported by our tool. A number of optimizations for improving indexing (parallel indexing), data subsetting (performance model) and data transfer (parallel streaming) are also applied. We have extensively evaluated our implementation. We compared our GridFTP SDQuery DSI with GridFTP default File DSI and showed that in different network environments, our method can achieve better efficiency in almost all cases.

Chair/Author Details:

Erwin Laure (Chair) - KTH Royal Institute of Technology

Yu Su - Ohio State University

Yi Wang - Ohio State University

Gagan Agrawal - Ohio State University

Rajkumar Kettimuthu - Argonne National Laboratory

  Click here to download .ics calendar file

  Click here to download .vcs calendar file

  Click here to add event to your Google Calendar

The full paper can be found in the ACM Digital Library