NDLI: A Scalable Reverse Lookup Scheme Using Group-Based Shifted Declustering Layout

Content Provider	IEEE Xplore Digital Library
Author	Junyao Zhang Pengju Shang Jun Wang
Copyright Year	2011
Abstract	Recent years have witnessed an increasing demand for super data clusters. The super data clusters have reached the petabyte-scale that can consist of thousands or tens of thousands storage nodes at a single site. For this architecture, reliability is becoming a great concern. In order to achieve a high reliability, data recovery and node reconstruction is a must. Although extensive research works have investigated how to sustain high performance and high reliability in case of node failures at large scale, a reverse lookup problem, namely finding the objects list for the failed node remains open. This is especially true for storage systems with high requirement of data integrity and availability, such as scientific research data clusters and etc. Existing solutions are either time consuming or expensive. Meanwhile, replication based block placement can be used to realize fast reverse lookup. However, they are designed for centralized, small-scale storage architectures. In this paper, we propose a fast and efficient reverse lookup scheme named Group-based Shifted Declustering (G-SD) layout that is able to locate the whole content of the failed node. G-SD extends our previous shifted declustering layout and applies to large-scale file systems. Our mathematical proofs and real-life experiments show that G-SD is a scalable reverse lookup scheme that is up to one order of magnitude faster than existing schemes.
Starting Page	604
Ending Page	615
File Size	1423329
Page Count	12
File Format	PDF
ISBN	9781612843728
ISSN	15302075
DOI	10.1109/IPDPS.2011.64
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2011-05-16
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Layout Peer to peer computing Redundancy Servers Heuristic algorithms Computer architecture
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Server selection with delay constraints for online games

A near-optimal one hop DHT lookup in structure peer-to-peer networks

An efficient lookup algorithm for dynamic peer-to-peer Chord

A scalable QoS-aware service aggregation model for peer-to-peer computing grids

Shifted declustering: a placement-ideal layout scheme for multi-way replication storage architecture

Shifted Declustering: A Placement-ideal Layout Scheme for Multi-way Replication Storage Architecture

Replica Placement Algorithms in hybrid CDN-P2P architectures

gMeasure: a group-based network performance measurement service for peer-to-peer applications

Hybrid CDN-P2P architecture: Replica content Placement Algorithms

A Scalable Reverse Lookup Scheme Using Group-Based Shifted Declustering Layout

Similar Documents

Server selection with delay constraints for online games

A near-optimal one hop DHT lookup in structure peer-to-peer networks

An efficient lookup algorithm for dynamic peer-to-peer Chord

A scalable QoS-aware service aggregation model for peer-to-peer computing grids

Shifted declustering: a placement-ideal layout scheme for multi-way replication storage architecture

Shifted Declustering: A Placement-ideal Layout Scheme for Multi-way Replication Storage Architecture

Replica Placement Algorithms in hybrid CDN-P2P architectures

gMeasure: a group-based network performance measurement service for peer-to-peer applications

Hybrid CDN-P2P architecture: Replica content Placement Algorithms

A Scalable Reverse Lookup Scheme Using Group-Based Shifted Declustering Layout