NDLI: Data locality in Hadoop cluster systems

Content Provider	IEEE Xplore Digital Library
Author	Khan, M. Yang Liu Maozhen Li
Copyright Year	2014
Description	Author affiliation: Sch. of Eng. & Design, Brunel Univ., Uxbridge, UK (Khan, M.; Maozhen Li) \|\| Sch. of Electr. Eng. & Inf. Syst., Sichuan Univ., Chengdu, China (Yang Liu)
Abstract	MapReduce has become a major programming model that supports distributed and parallel processing for large-scale data-intensive applications such as Web data mining, network traffic analysis, machine learning and scientific simulation. Hadoop is the most popular open-source implementation of the MapReduce programming model. In Hadoop, input files are divided into many data blocks and these blocks are distributed over several nodes in cluster. To efficiently process the data blocks, Hadoop should provide an efficient scheduling mechanism for enhancing the performance of the system in a shared cluster environment. In Hadoop scheduling mainly caused by data locality issues due to limited network bandwidth. By introducing the scheduling issues with regarding to the data locality, this paper review different data locality aware scheduling algorithms that handling the data locality issues. In addition, this paper also evaluating their features, strength, weakness and provided some guidelines on how to improve further these scheduling algorithms.
Starting Page	720
Ending Page	724
File Size	211534
Page Count	5
File Format	PDF
e-ISBN	9781479951482
DOI	10.1109/FSKD.2014.6980924
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-08-19
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Scheduling algorithms Prefetching data locality Distributed databases Clustering algorithms Scheduling Delays job scheduling MapReduce
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in