SC13 Denver, CO

The International Conference for High Performance Computing, Networking, Storage and Analysis

A Lightweight Data Location Service for Nondeterministic Exascale Storage Systems.

Authors: Zhiwei Sun (University of Alabama at Birmingham), Anthony Skjellum (University of Alabama at Birmingham), Lee Ward (Sandia National Laboratories), Matthew Curry (Sandia National Laboratories)

Abstract: We present LWDLS, a lightweight data location service designed for Exascale storage systems (storage systems with order of 10^18 bytes) and geo-distributed storage systems (large storage systems with physically distributed locations). LWDLS provides a search-based data location solution, and enables free data placement, movement, and replication. Probe and prune protocols are introduced that reduce topology mismatch, and a heuristic flooding search algorithm (HFS) is presented with higher search efficiency than pure flooding search while delivering comparable search speed and coverage to pure flooding search. LWDLS is lightweight and scalable in terms of incorporating low overhead, high search efficiency, no global state, and by avoiding periodic messages. LWDLS is fully distributed and can be used in nondeterministic storage systems and in deterministic storage systems. Extensive simulations modeling large-scale High Performance Computing (HPC) storage environments provide representative performance outcomes.

Poster: pdf
Two-page extended abstract: pdf

Poster Index