SCHEDULE: NOV 16-22, 2013
When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.
Algorithms for High-Throughput Disk-to-Disk Sorting
SESSION: Sorting and Graph Algorithms
EVENT TYPE: Papers
TIME: 2:30PM - 3:00PM
SESSION CHAIR: Karen Devine
AUTHOR(S):Hari Sundar, Dhairya Malhotra, Karl Schulz
ROOM:205/207
ABSTRACT:
In this paper, we present a new out-of-core sort algorithm, designed for problems that are too large to fit into the aggregate RAM available on modern supercomputers. We analyze the performance including the cost of IO and demonstrate the fastest reported throughput using the canonical sortBenchmark on a general-purpose, production HPC resource running Lustre. By clever use of available storage and a formulation of asynchronous data transfer mechanisms, we are able to almost completely hide the computation (sorting) behind the IO latency. This latency hiding enables us to achieve comparable execution times, including the additional temporary IO required, between a large sort problem (5TB) run as a single, in-RAM sort and our out-of-core approach using 1/10th the amount of RAM. In our largest run, sorting 100TB of records using 1792-hosts, we achieved an end-to-end throughput of 1.24TB/min using our general-purpose sorter, improving on the current Daytona record holder by 65%.
Chair/Author Details:
Karen Devine (Chair) - Sandia National Laboratories
Hari Sundar - University of Texas at Austin
Dhairya Malhotra - University of Texas at Austin
Karl Schulz - University of Texas at Austin
Click here to download .ics calendar file
Click here to download .vcs calendar file
Click here to add event to your Google Calendar
The full paper can be found in the ACM Digital Library