NDLI: Extreme Binning: Scalable, parallel deduplication for chunk-based file backup

Content Provider	IEEE Xplore Digital Library
Author	Bhagwat, D. Eshghi, K. Long, D.D.E. Lillibridge, M.
Copyright Year	2009
Description	Author affiliation: University of California, 1156 High Street, Santa Cruz, CA 95064 (Bhagwat, D.) \|\| University of California, 1156 High Street Santa Cruz, CA 95064 (Long, D.D.E.) \|\| Hewlett-Packard Laboratories, 1501 Page Mill Rd, Palo Alto, CA 94304 (Eshghi, K.; Lillibridge, M.)
Abstract	Data deduplication is an essential and critical component of backup systems. Essential, because it reduces storage space requirements, and critical, because the performance of the entire backup operation depends on its throughput. Traditional backup workloads consist of large data streams with high locality, which existing deduplication techniques require to provide reasonable throughput. We present Extreme Binning, a scalable deduplication technique for non-traditional backup workloads that are made up of individual files with no locality among consecutive files in a given window of time. Due to lack of locality, existing techniques perform poorly on these workloads. Extreme Binning exploits file similarity instead of locality, and makes only one disk access for chunk lookup per file, which gives reasonable throughput. Multi-node backup systems built with Extreme Binning scale gracefully with the amount of input data; more backup nodes can be added to boost throughput. Each file is allocated using a stateless routing algorithm to only one node, allowing for maximum parallelization, and each backup node is autonomous with no dependency across nodes, making data management tasks robust with low overhead.
Starting Page	1
Ending Page	9
File Size	326340
Page Count	9
File Format	PDF
ISBN	9781424449279
ISSN	15267539
DOI	10.1109/MASCOT.2009.5366623
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-09-21
Publisher Place	United Kingdom
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Throughput Intrusion detection Laboratories Milling machines Space technology Routing Robustness Web pages Digital images Electronic mail
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Extreme binning: Scalable, parallel deduplication for chunk-based file backup (2009)

Extreme binning: Scalable, parallel deduplication for chunk-based file backup (2009)

Extreme Binning: Scalable, Parallel Deduplication for Chunk-based File Backup

Improved deduplication through parallel binning.

Identification of Malicious Web Pages with Static Heuristics

MAD2: A scalable high-throughput exact deduplication approach for network backup services

Similarity Based Deduplication with Small Data Chunks

Design of a Web-based education environment

Sampling and locality.

Extreme Binning: Scalable, parallel deduplication for chunk-based file backup

Similar Documents

Extreme binning: Scalable, parallel deduplication for chunk-based file backup (2009)

Extreme binning: Scalable, parallel deduplication for chunk-based file backup (2009)

Extreme Binning: Scalable, Parallel Deduplication for Chunk-based File Backup

Improved deduplication through parallel binning.

Identification of Malicious Web Pages with Static Heuristics

MAD2: A scalable high-throughput exact deduplication approach for network backup services

Similarity Based Deduplication with Small Data Chunks

Design of a Web-based education environment

Sampling and locality.

Extreme Binning: Scalable, parallel deduplication for chunk-based file backup