NDLI: Semantic characterization of MapReduce workloads

Content Provider	IEEE Xplore Digital Library
Author	Zhihong Xu Hirzel, M. Rothermel, G.
Copyright Year	2013
Description	Author affiliation: IBM Res., Yorktown Heights, NY, USA (Hirzel, M.) \|\| Univ. of Nebraska, Lincoln, NE, USA (Zhihong Xu; Rothermel, G.)
Abstract	MapReduce is a platform for analyzing large amounts of data on clusters of commodity machines. MapReduce is popular, in part thanks to its apparent simplicity. However, there are unstated requirements for the semantics of MapReduce applications that can affect their correctness and performance. MapReduce implementations do not check whether user code satisfies these requirements, leading to time-consuming debugging sessions, performance problems, and, worst of all, silently corrupt results. This paper makes these requirements explicit, framing them as semantic properties and assumed outcomes. It describes a black-box approach for testing for these properties, and uses the approach to characterize the semantics of 23 non-trivial MapReduce workloads. Surprisingly, we found that for most requirements, there is at least one workload that violates it. This means that MapReduce may be simple to use, but it is not as simple to use correctly. Based on our results, we provide insights to users on how to write higher-quality MapReduce code, and insights to system and language designers on ways to make their platforms more robust.
Sponsorship	IEEE Comput. Soc.
Starting Page	87
Ending Page	97
File Size	430396
Page Count	11
File Format	PDF
ISBN	9781479905539
e-ISBN	9781479905553
DOI	10.1109/IISWC.2013.6704673
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-09-22
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Semantics Testing Context Commutation Educational institutions Debugging Fault tolerance
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Towards Dependability Testing of MapReduce Systems

Adding context to fault localization with integration coverage

Semantic characterization of mapreduce workloads (2013)

MARLA: MapReduce for Heterogeneous Clusters

MARIANE: MApReduce Implementation Adapted for HPC Environments

MapReduce Model Implementation on MPI Platform

Using Mutants to Locate "Unknown" Faults

ReCT: Improving MapReduce performance under failures with resilient checkpointing tactics

Dynamic publish/subscribe systems

Semantic characterization of MapReduce workloads

Similar Documents

Towards Dependability Testing of MapReduce Systems

Adding context to fault localization with integration coverage

Semantic characterization of mapreduce workloads (2013)

MARLA: MapReduce for Heterogeneous Clusters

MARIANE: MApReduce Implementation Adapted for HPC Environments

MapReduce Model Implementation on MPI Platform

Using Mutants to Locate "Unknown" Faults

ReCT: Improving MapReduce performance under failures with resilient checkpointing tactics

Dynamic publish/subscribe systems

Semantic characterization of MapReduce workloads