NDLI: Taming Latency in Data Center Networking with Erasure Coded Files

Content Provider	IEEE Xplore Digital Library
Author	Yu Xiang Aggarwal, V. Chen, Y.-F.R. Tian Lan
Copyright Year	2015
Description	Author affiliation: Dept. of ECE, George Washington Univ., Washington, DC, USA (Yu Xiang; Tian Lan) \|\| AT&T Labs.-Res., Bedminster, NJ, USA (Chen, Y.-F.R.) \|\| Sch. of IE, Purdue Univ., West Lafayette, IN, USA (Aggarwal, V.)
Abstract	This paper proposes an approach to minimize service latency in a data center network where erasure-coded files are stored on distributed disks/racks and access requests are scattered across the network. Due to limited bandwidth available at both top-of-the-rack and aggregation switches, network bandwidth must be apportioned among different intra-and inter-rack data flows in line with their traffic statistics. We formulate this problem as weighted queuing and employ a class of probabilistic request scheduling policies to derive a closed-form outer-bound of service latency for erasure-coded storage with arbitrary file access patterns and service time distributions. The result enables us to propose a joint latency optimization over three entangled "control knobs": the bandwidth allocation at top-of-the-rack and aggregation switches, the probabilities for scheduling file requests, and the placement of encoded file chunks, which affects data locality. The joint optimization is shown to be a mixed-integer problem. We develop an iterative algorithm which decouples and solves the joint optimization as three sub-problems, which are either convex or solvable via bipartite matching in polynomial time. The proposed algorithm is prototyped in an open-source, distributed file system, Tahoe, and evaluated on a cloud tested with 16 separate physical hosts in an Open Stack cluster. Experiments validate our theoretical latency analysis and show significant latency reduction for diverse file access patterns. The results provide valuable insight on designing low-latency data center networks with erasure-coded storage.
Starting Page	241
Ending Page	250
File Size	958877
Page Count	10
File Format	PDF
ISBN	9781479980062
DOI	10.1109/CCGrid.2015.142
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2015-05-04
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Bandwidth Delays Optimization Servers Aggregates Joints Upper bound service latency Erasure-coded storage data center
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Use of erasure code for low latency cloud storage

Multi-tenant Latency Optimization in Erasure-Coded Storage with Differentiated Services

Joint latency and cost optimization for erasure-coded data center storage,” 2014. [Online]. Available: http://arxiv.org/abs/1404.4975

An Erasure Coded Archival Storage System

Joint Host-Network Optimization for Energy-Efficient Data Center Networking

Differentiated latency in data center networks with erasure coded files through traffic engineering

Taming Tail Latency for Erasure-coded, Distributed Storage Systems

LaRS: A Load-Aware Recovery Scheme for Heterogeneous Erasure-Coded Storage Clusters

Building a network coded storage testbed for data center energy reduction

Taming Latency in Data Center Networking with Erasure Coded Files

Similar Documents

Use of erasure code for low latency cloud storage

Multi-tenant Latency Optimization in Erasure-Coded Storage with Differentiated Services

Joint latency and cost optimization for erasure-coded data center storage,” 2014. [Online]. Available: http://arxiv.org/abs/1404.4975

An Erasure Coded Archival Storage System

Joint Host-Network Optimization for Energy-Efficient Data Center Networking

Differentiated latency in data center networks with erasure coded files through traffic engineering

Taming Tail Latency for Erasure-coded, Distributed Storage Systems

LaRS: A Load-Aware Recovery Scheme for Heterogeneous Erasure-Coded Storage Clusters

Building a network coded storage testbed for data center energy reduction

Taming Latency in Data Center Networking with Erasure Coded Files