NDLI: LoadAtomizer: A locality and I/O load aware task scheduler for MapReduce

Content Provider	IEEE Xplore Digital Library
Author	Asahara, M. Nakadai, S. Araki, T.
Copyright Year	2012
Description	Author affiliation: Cloud System Research Laboratories NEC Corporation Kawasaki, Kanagawa 211-8666, Japan (Asahara, M.; Nakadai, S.; Araki, T.)
Abstract	Data-intensive computing systems like MapReduce and Dryad have emerged as a framework for leveraging computing resources of a cluster. I/O bottlenecks need to be eased to improve performance in data-intensive computing systems. State-of-the-art frameworks for data-intensive computing have tackled the issue with a data locality based task scheduling policy. However, locality-aware scheduling does not always work good to mitigate I/O bottlenecks when different I/O characteristic jobs run concurrently. This paper presents LoadAtomizer, a locality and I/O load aware task scheduler for MapReduce. LoadAtomizer mitigates the I/O bottlenecks of a cluster with locality and I/O load aware map task assignment and storage selection. LoadAtomizer quickly assigns a slave a map task whose input data is stored in a lightly loaded storage and commands the slave to read the input data from the storage. LoadAtomizer maintains the load information of storages and the network with a topology-aware load tree. A topology-aware load tree enables LoadAtomizer to select quickly a lightly loaded storage that a slave can access through a lightly loaded network path. Experimental results demonstrated that our prototype of LoadAtomizer shortened completion time of multiple jobs by up to 18.6 %.
Starting Page	317
Ending Page	324
File Size	554066
Page Count	8
File Format	PDF
ISBN	9781467345118
e-ISBN	9781467345101
e-ISBN	9781467345095
DOI	10.1109/CloudCom.2012.6427572
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-12-03
Publisher Place	Taiwan
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	I/O Control Cloud computing Maplceduce Processor scheduling Conferences Switches Bandwidth Task Scheduling Throughput Scheduling
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in