SC13 Home > SC13 Schedule > SC13 Presentation - CooMR: Cross-Task Coordination for Efficient Data Management in MapReduce Programs

SCHEDULE: NOV 16-22, 2013

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

CooMR: Cross-Task Coordination for Efficient Data Management in MapReduce Programs

SESSION: System-wide Application Performance Assessments


TIME: 4:30PM - 5:00PM

SESSION CHAIR: Dominik Ulmer

AUTHOR(S):Xiaobing Li, Yandong Wang, YIzheng Jiao, Cong Xu, Weikuan Yu


Hadoop is a popular open-source implementation of a MapReduce programming model for big data processing. It represents system resources as map and reduce slots and schedules them to various tasks. This execution model gives little regard to the need of cross-task coordination on the use of shared resources on each node, which results in task interference. In addition, the existing merge algorithm causes excessive I/O. In this study, we undertake an effort to address both issues. Accordingly, we introduce a cross-task coordination framework called CooMR for efficient data management in MapReduce programs. CooMR consists of three component schemes including cross-task opportunistic memory sharing and log-structured I/O consolidation, which aim to facilitate task coordination, and a key-based in-situ merge algorithm designed to enable the sorting/merging of intermediate data without actually moving the pairs. Our evaluation demonstrates that CooMR can increase task coordination, improve resource utilization, and effectively accelerate MapReduce programs.

Chair/Author Details:

Dominik Ulmer (Chair) - Cray Switzerland

Xiaobing Li - Auburn University

Yandong Wang - Auburn University

YIzheng Jiao - Auburn University

Cong Xu - Auburn University

Weikuan Yu - Auburn University

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

The full paper can be found in the ACM Digital Library