NDLI: Tree-structured data regeneration with network coding in distributed storage systems

Content Provider	IEEE Xplore Digital Library
Author	Jun Li Shuang Yang Xin Wang Xiangyang Xue Baochun Li
Copyright Year	2009
Description	Author affiliation: Department of Electrical and Computer Engineering, University of Toronto, Canada (Baochun Li) \|\| School of Computer Science, Fudan University, China (Jun Li; Shuang Yang; Xin Wang; Xiangyang Xue)
Abstract	Distributed storage systems, built on peer-to-peer networks, can provide large-scale data storage and high data reliability by redundant schemes, such as replica, erasure codes and linear network coding. Redundant data may get lost due to the instability of distributed systems, such as permanent node departures, hardware failures, and accidental deletions. In order to maintain data availability, it is necessary to regenerate new redundant data in another node, referred to as a newcomer. Regeneration is expected to be finished as soon as possible, because the regeneration time can influence the data reliability and availability of distributed storage systems. It has been acknowledged that linear network coding can regenerate redundant data with less network traffic than replica and erasure codes. However, previous regeneration schemes are all star-structured regeneration schemes, in which data are transferred directly from existing storage nodes, referred to as providers, to the newcomer, so the regeneration time is always limited by the path with the narrowest bandwidth between newcomer and provider, due to bandwidth heterogeneity. In this paper, we exploit the bandwidth between providers and propose a tree-structured regeneration scheme using linear network coding. In our scheme, data can be transferred from providers to the newcomer through a regeneration tree, defined as a spanning tree covering the newcomer and all the providers. In a regeneration tree, a provider can receive data from other providers, then encode the received data with the data this provider stores, and finally send the encoded data to another provider or to the newcomer. We prove that a maximum spanning tree is an optimal regeneration tree and analyze its performance. In a trace-based simulation, the results show the tree-structured scheme can reduce the regeneration time by 75%–82% and improve data availability by 73%–124%.
Starting Page	1
Ending Page	9
File Size	665593
Page Count	9
File Format	PDF
ISBN	9781424438754
ISSN	1548615X
e-ISBN	9781424438761
DOI	10.1109/IWQoS.2009.5201391
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-07-13
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Network coding Peer to peer computing Availability Bandwidth Large-scale systems Memory Hardware Maintenance Telecommunication traffic Performance analysis Maximum Spanning Tree Distributed Storage System Linear Network Coding
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Tree-structured Data Regeneration in Distributed Storage Systems with Regenerating Codes

Notice of RetractionZipf's Trust Discovery in Structured P2P Network

Distributed spanning tree algorithms for large scale traversals

Tree-structured data regeneration with network coding in distributed storage systems.

Randomized network coding in distributed storage systems with layered overlay

MCR: A mutual cooperative recovery mechanism in peer-to-peer storage systems

Node recovery schemes for minimizing repair time in distributed storage system with network coding

Cooperative Recovery of Distributed Storage Systems from Multiple Losses with Network Coding

Distributed storage systems and connection to network coding

Tree-structured data regeneration with network coding in distributed storage systems

Similar Documents

Tree-structured Data Regeneration in Distributed Storage Systems with Regenerating Codes

Notice of RetractionZipf's Trust Discovery in Structured P2P Network

Distributed spanning tree algorithms for large scale traversals

Tree-structured data regeneration with network coding in distributed storage systems.

Randomized network coding in distributed storage systems with layered overlay

MCR: A mutual cooperative recovery mechanism in peer-to-peer storage systems

Node recovery schemes for minimizing repair time in distributed storage system with network coding

Cooperative Recovery of Distributed Storage Systems from Multiple Losses with Network Coding

Distributed storage systems and connection to network coding

Tree-structured data regeneration with network coding in distributed storage systems