NDLI: Deadline-Aware Scheduling and Flexible Bandwidth Allocation for Big-Data Transfers

Content Provider	Directory of Open Access Journals (DOAJ)
Author	Srinikethan Madapuzi Srinivasan Tram Truong-Huu Mohan Gurusamy
Abstract	Big data is becoming a major focus for both industry and academia, requiring drastic changes in all aspects of computer systems in order to store, process, and transfer big data. In networks, a fundamental problem is how to efficiently transfer big data since the performance is affected by several factors such as path, bandwidth, and scheduled start time. Best-effort algorithms are no longer applicable as they may not satisfy the deadline requirement of the requests. In this paper, we consider the problem of scheduling and flexible bandwidth allocation for big-data transfers with deadline constraints. With flexible bandwidth allocation, the bandwidth allocated to a request can be dynamically adjusted any time during its transfer. We develop an optimization programming formulation that provides admission and scheduling decisions, bandwidth allocation, and path selection for each accepted request. The formulation aims at maximizing the acceptance while guaranteeing the deadline constraints of transfer requests. Due to the complex nature of the optimization problem, we develop a two-phase heuristic algorithm namely deadline-aware flexible bandwidth allocation for big-data transfers (DaFBA). We develop two scheduling approaches for DaFBA using batch scheduling to be used for every time interval and dynamic scheduling to be used upon every request arrival. We evaluate the performance of the proposed algorithm through comprehensive simulations with two routing scenarios: pre-computed path scenario and load-based routing scenario. The results show that the proposed algorithm performs close to the optimal solution and outperforms baseline algorithms in terms of rejection ratio and the amount of data transferred.
e-ISSN	21693536
DOI	10.1109/ACCESS.2018.2882877
Journal	IEEE Access
Volume Number	6
Language	English
Publisher	IEEE
Publisher Date	2018-01-01
Publisher Place	United States
Access Restriction	Open
Subject Keyword	Electrical Engineering. Electronics. Nuclear Engineering Big-data Transfers Flexible Bandwidth Allocation Load-based Routing Transfer Deadlines Traffic Flow Scheduling
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in