NDLI: On channel failures, file fragmentation policies, and heavy-tailed completion times

Content Provider	ACM Digital Library
Author	Low, Steven H. Andrew, Lachlan L. H. Doyle, John C. Nair, Jayakrishnan Andreasson, Martin
Copyright Year	2016
Abstract	It has been recently discovered that heavy-tailed completion times can result from protocol interaction even when file sizes are light-tailed. A key to this phenomenon is the use of a restart policy where if the file is interrupted before it is completed, it needs to restart from the beginning. In this paper, we show that fragmenting a file into pieces whose sizes are either bounded or independently chosen after each interruption guarantees light-tailed completion time as long as the file size is light-tailed; i.e., in this case, heavy-tailed completion time can only originate from heavy-tailed file sizes. If the file size is heavy-tailed, then the completion time is necessarily heavy-tailed. For this case, we show that when the file size distribution is regularly varying, then under independent or bounded fragmentation, the completion time tail distribution function is asymptotically bounded above by that of the original file size stretched by a constant factor. We then prove that if the distribution of times between interruptions has nondecreasing failure rate, the expected completion time is minimized by dividing the file into equal-sized fragments; this optimal fragment size is unique but depends on the file size. We also present a simple blind fragmentation policy where the fragment sizes are constant and independent of the file size and prove that it is asymptotically optimal. Both these policies are also shown to have desirable completion time tail behavior. Finally, we bound the error in expected completion time due to error in modeling of the failure process.
Starting Page	529
Ending Page	541
Page Count	13
File Format	PDF
ISSN	10636692
DOI	10.1109/TNET.2014.2375920
Volume Number	24
Issue Number	1
Journal	IEEE/ACM Transactions on Networking (TON)
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-02-01
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Checkpointing File fragmentation Heavy tails Optimal packet size Server failure
Content Type	Text
Resource Type	Article
Subject	Electrical and Electronic Engineering Computer Networks and Communications Software Computer Science Applications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

On Channel Failures, File Fragmentation Policies, and Heavy-Tailed Completion Times

1On Channel Failures, File Fragmentation Policies, and Heavy-Tailed Completion Times

File Fragmentation over an Unreliable Channel (2010)

File fragmentation over an unreliable channel

Dynamic packet fragmentation for wireless channels with failures

Delay Stability of Back-Pressure Policies in the Presence of Heavy-Tailed Traffic

Max-Weight Scheduling in Queueing Networks With Heavy-Tailed Traffic

Max-Weight Scheduling in Queueing Networks With Heavy-Tailed Traffic

On checkpointing and heavy-tails in unreliable computing environments

On channel failures, file fragmentation policies, and heavy-tailed completion times

Similar Documents

On Channel Failures, File Fragmentation Policies, and Heavy-Tailed Completion Times

1On Channel Failures, File Fragmentation Policies, and Heavy-Tailed Completion Times

File Fragmentation over an Unreliable Channel (2010)

File fragmentation over an unreliable channel

Dynamic packet fragmentation for wireless channels with failures

Delay Stability of Back-Pressure Policies in the Presence of Heavy-Tailed Traffic

Max-Weight Scheduling in Queueing Networks With Heavy-Tailed Traffic

Max-Weight Scheduling in Queueing Networks With Heavy-Tailed Traffic

On checkpointing and heavy-tails in unreliable computing environments

On channel failures, file fragmentation policies, and heavy-tailed completion times