NDLI: Joint Scheduling of Data and Computation in Geo-Distributed Cloud Systems

Content Provider	IEEE Xplore Digital Library
Author	Lingyan Yin Jizhou Sun Laiping Zhao Chenzhou Cui Jian Xiao Ce Yu
Copyright Year	2015
Description	Author affiliation: Sch. of Comput. Sci. & Technol., Tianjin Univ., Tianjin, China (Lingyan Yin; Jizhou Sun; Ce Yu) \|\| Sch. of Comput. Software, Tianjin Univ., Tianjin, China (Laiping Zhao; Jian Xiao) \|\| Nat. Astron. Obs., NAOC, Beijing, China (Chenzhou Cui)
Abstract	Recent trends show that cloud computing is growing to span more and more globally distributed data centers. For geo-distributed data centers, there is an increasing need for scheduling algorithms to place tasks across data centers, by jointly considering data and computation. This scheduling must deal with situations such as wide-area distributed data, data sharing, WAN bandwidth costs and data center capacity limits, while also minimizing completion time. However, this kind of scheduling problems is known to be NP-Hard. In this paper, inspired by real applications in astronomy field, we propose a two-phase scheduling algorithm that addresses these challenges. The mapping phase groups tasks considering the data-sharing relations, and dispatches groups to data centers by way of one-to-one correspondence. The reassigning phase balances the completion time across data centers according to relations between tasks and groups. We utilize the real China-Astronomy-Cloud model and typical applications to evaluate our proposal. Simulations show that our algorithm obtains up to 22% better completion time and effectively reduces the amount of data transfers compared with other similar scheduling algorithms.
Starting Page	657
Ending Page	666
File Size	1025374
Page Count	10
File Format	PDF
ISBN	9781479980062
DOI	10.1109/CCGrid.2015.83
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2015-05-04
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Data transfer Computational modeling Distributed databases Scheduling algorithms Astronomy Data models cloud computing geo-distributed data centers scheduling algorithms data and computation intensive
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A Scheduling Framework for Periodic Tasks in Geo-Distributed Data Centers

Nebula: Distributed Edge Cloud for Data Intensive Computing

SchedCD: A scheduling algorithm for cloud computing in data centers

SAGE: Geo-Distributed Streaming Data Analysis in Clouds

FSBD: A Framework for Scheduling of Big Data Mining in Cloud Computing

A scheduling mechanism for multiple MapReduce jobs in a workflow application (position paper)

BIGhybrid -- A Toolkit for Simulating MapReduce in Hybrid Infrastructures

Data Placement and Task Scheduling Optimization for Data Intensive Scientific Workflow in Multiple Data Centers Environment

Smart partitioning of geo-distributed resources to improve cloud network performance

Joint Scheduling of Data and Computation in Geo-Distributed Cloud Systems

Similar Documents

A Scheduling Framework for Periodic Tasks in Geo-Distributed Data Centers

Nebula: Distributed Edge Cloud for Data Intensive Computing

SchedCD: A scheduling algorithm for cloud computing in data centers

SAGE: Geo-Distributed Streaming Data Analysis in Clouds

FSBD: A Framework for Scheduling of Big Data Mining in Cloud Computing

A scheduling mechanism for multiple MapReduce jobs in a workflow application (position paper)

BIGhybrid -- A Toolkit for Simulating MapReduce in Hybrid Infrastructures

Data Placement and Task Scheduling Optimization for Data Intensive Scientific Workflow in Multiple Data Centers Environment

Smart partitioning of geo-distributed resources to improve cloud network performance

Joint Scheduling of Data and Computation in Geo-Distributed Cloud Systems