NDLI: DRAW: A New Data-gRouping-AWare Data Placement Scheme for Data Intensive Applications With Interest Locality

Content Provider	IEEE Xplore Digital Library
Author	Wang, J. Xiao, Q. Yin, J. Shang, P.
Copyright Year	1965
Abstract	Recent years have seen an increasing number of scientists employ data parallel computing frameworks such as MapReduce and Hadoop to run data intensive applications and conduct analysis. In these co-located compute and storage frameworks, a wise data placement scheme can significantly improve the performance. Existing data parallel frameworks, e.g., Hadoop, or Hadoop-based clouds, distribute the data using a random placement method for simplicity and load balance. However, we observe that many data intensive applications exhibit interest locality which only sweep part of a big data set. The data often accessed together result from their grouping semantics. Without taking data grouping into consideration, the random placement does not perform well and is way below the efficiency of optimal data distribution. In this paper, we develop a new Data-gRouping-AWare (DRAW) data placement scheme to address the above-mentioned problem. DRAW dynamically scrutinizes data access from system log files. It extracts optimal data groupings and re-organizes data layouts to achieve the maximum parallelism per group subjective to load balance. By experimenting two real-world MapReduce applications with different data placement schemes on a 40-node test bed, we conclude that DRAW increases the total number of local map tasks executed up to 59.8%, reduces the completion latency of the map phase up to 41.7%, and improves the overall performance by 36.4%, in comparison with Hadoop's default random placement.
Sponsorship	IEEE Magnetics Society
Starting Page	2514
Ending Page	2520
Page Count	7
File Size	1260930
File Format	PDF
ISSN	00189464
Volume Number	49
Issue Number	6
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-06-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	MapReduce Data intensive data layout Hadoop
Content Type	Text
Resource Type	Article
Subject	Electronic, Optical and Magnetic Materials Electrical and Electronic Engineering

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Optimizing Crawler4j using MapReduce Programming Model

Expanded Assessment of a Practical Thermally Aware Energy-Optimized Load Placement Strategy for Open-Aisle, Air-Cooled Data Centers

Thermally Aware, Energy-Based Load Placement in Open-Aisle, Air-Cooled Data Centers

Mammoth: Gearing Hadoop Towards Memory-Intensive MapReduce Applications

A MapReduce task scheduling algorithm for deadline constraints

Dynamic Load Aware MapReduce Task Scheduler for Cloud Environment

Structure-Aware Placement Techniques for Designs With Datapaths

Optimization of Data Center Room Layout to Minimize Rack Inlet Air Temperature

Simulation for a Shingled Magnetic Recording Disk

DRAW: A New Data-gRouping-AWare Data Placement Scheme for Data Intensive Applications With Interest Locality

Similar Documents

Optimizing Crawler4j using MapReduce Programming Model

Expanded Assessment of a Practical Thermally Aware Energy-Optimized Load Placement Strategy for Open-Aisle, Air-Cooled Data Centers

Thermally Aware, Energy-Based Load Placement in Open-Aisle, Air-Cooled Data Centers

Mammoth: Gearing Hadoop Towards Memory-Intensive MapReduce Applications

A MapReduce task scheduling algorithm for deadline constraints

Dynamic Load Aware MapReduce Task Scheduler for Cloud Environment

Structure-Aware Placement Techniques for Designs With Datapaths

Optimization of Data Center Room Layout to Minimize Rack Inlet Air Temperature

Simulation for a Shingled Magnetic Recording Disk

DRAW: A New Data-gRouping-AWare Data Placement Scheme for Data Intensive Applications With Interest Locality