NDLI: Reducing MapReduce Abstraction Costs for Text-centric Applications

Content Provider	IEEE Xplore Digital Library
Author	Chun-Hung Hsiao Cafarella, M. Narayanasamy, S.
Copyright Year	2014
Description	Author affiliation: Univ. of Michigan, Ann Arbor, MI, USA (Chun-Hung Hsiao; Cafarella, M.; Narayanasamy, S.)
Abstract	The MapReduce framework has become widely popular for programming large clusters, even though MapReduce jobs may use underlying resources relatively inefficiently. There has been substantial research in improving MapReduce performance for applications that were inspired by relational database queries, but almost none for text-centric applications, including inverted index construction, processing large log files, and so on. We identify two simple optimizations to improve MapReduce performance on text-centric tasks: frequency-buffering and spill-matcher. The former approach improves buffer efficiency for intermediate map outputs by identifying frequent keys, effectively shrinking the amount of work that the shuffle phase must perform. Spill-matcher is a runtime controller that improves parallelization of MapReduce framework background tasks. Together, our two optimizations improve the performance of text-centric applications by up to 39.1%. We demonstrate gains on both a small local cluster and Amazon's EC2 cloud service. Unlike other MapReduce optimizations, these techniques require no user code changes, and only small changes to the MapReduce system.
Starting Page	40
Ending Page	49
File Size	425518
Page Count	10
File Format	PDF
ISBN	9781479956180
ISSN	01903918
DOI	10.1109/ICPP.2014.13
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-09-09
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Optimization Parallel processing Instruction sets Sorting Standards Indexes Runtime
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Rearchitecting MapReduce for Heterogeneous Multicore Processors with Explicitly Managed Memories

Sorting sixteen numbers

Can GPUs sort strings efficiently?

Parallel processing algorithm for Bayesian network inference

Reducing lock contention on multi-core platforms

Runtime Overhead Reduction in Automated Parallel Processing System Using Valgrind

MapReuse: Reusing Computation in an In-Memory MapReduce System

Thread-level speculation as an optimization technique in Web Applications — Initial results

A Multimedia Parallel Processing Approach on GPU MapReduce Framework

Reducing MapReduce Abstraction Costs for Text-centric Applications

Similar Documents

Rearchitecting MapReduce for Heterogeneous Multicore Processors with Explicitly Managed Memories

Sorting sixteen numbers

Can GPUs sort strings efficiently?

Parallel processing algorithm for Bayesian network inference

Reducing lock contention on multi-core platforms

Runtime Overhead Reduction in Automated Parallel Processing System Using Valgrind

MapReuse: Reusing Computation in an In-Memory MapReduce System

Thread-level speculation as an optimization technique in Web Applications — Initial results

A Multimedia Parallel Processing Approach on GPU MapReduce Framework

Reducing MapReduce Abstraction Costs for Text-centric Applications