NDLI: Leveraging data deduplication to improve the performance of primary storage systems in the cloud

Please wait, while we are loading the content...

Pico replication: a high availability framework for middleboxes

Hierarchical scheduling for diverse datacenter workloads

Small is better: avoiding latency traps in virtualized data centers

Orbe: scalable causal consistency using dependency matrices and physical clocks

Limplock: understanding the impact of limpware on scale-out cloud systems

Scalable lineage capture for debugging DISC analytics

EventWave: programming model and runtime support for tightly-coupled elastic cloud applications

Leveraging data deduplication to improve the performance of primary storage systems in the cloud

On fault resilience of OpenStack

Apache Hadoop YARN: yet another resource negotiator

Go, server, go!: parallel computing with moving servers

Leveraging sharding in the design of scalable replication protocols

When the network crumbles: an empirical study of cloud network failures and their impact on services

Memory-efficient groupby-aggregate using compressed buffer trees

Tolerating business failures in hosted applications

FAST: near real-time data analytics for the cloud

COLO: COarse-grained LOck-stepping virtual machines for non-stop service

Natjam: design and evaluation of eviction policies for supporting priorities and deadlines in mapreduce clusters

Virtual network diagnosis as a service

Understanding and mitigating the impact of load imbalance in the memory caching tier

vTube: efficient streaming of virtual appliances over last-mile networks

Memory footprint matters: efficient equi-join algorithms for main memory data processing

Consistency without borders

Towards a performance-as-a-service cloud

jVerbs: ultra-low latency for data center applications

Scale-up vs scale-out for Hadoop: time to rethink?

MRM: delivering predictability and service differentiation in shared compute clusters

Client-centric benchmarking of eventual consistency for cloud storage systems

Introducing service-level awareness in the cloud

Mortar: filling the gaps in data center memory

PoWER: prediction of workload for energy efficient relocation of virtual machines

Simple and efficient coupling of Hadoop with a database engine

The wisdom of virtual crowds: mining datacenter telemetry to collaboratively debug performance

Extending modern PaaS clouds with BSP to execute legacy MPI applications

Adaptive partitioning for large-scale dynamic graphs

Towards a general framework for secure MapReduce computation on hybrid clouds

CATS: a linearizable and self-organizing key-value store

FastMR: fast processing for large distributed data streams

XDB: a novel database architecture for data analytics as a service

CloudLEGO: scalable cross-VM-type application performance prediction

Comparing SSD-placement strategies to scale a database-in-the-cloud

VMShadow: optimizing the performance of virtual desktops in distributed clouds

Fault-tolerant industrial automation as a cloud service

High performance clustering of social images in a map-collective programming model

Does RDMA-based enhanced Hadoop MapReduce need a new performance model?

Syndicate: democratizing cloud storage and caching through service composition

Coloring the cloud for predictable performance

Decentralized privacy protection strategies for location-based services

Recommending just enough memory for analytics

Process-oriented recovery for operations on cloud applications

CloudSSI: revisiting SSI in cloud era

Firewall placement in cloud data centers

Harmony: coordinating network, compute, and storage in software-defined clouds

Pregelix: dataflow-based big graph analytics

Wide-area streaming analytics: distributing the data cube

High performance in-memory caching through flexible fine-grained services

Compiling machine learning algorithms with SystemML

SuperCloud: economical cloud service on multiple vendors

Dynamic performance profiling of cloud caches

DEDIS: distributed exact deduplication for primary storage infrastructures

Leveraging data deduplication to improve the performance of primary storage systems in the cloud

Content Provider	ACM Digital Library
Author	Tian, Lei Jiang, Hong Wu, Suzhen Mao, Bo
Abstract	Recent studies have shown that moderate to high data redundancy exists in primary storage systems, such as VM-based, enterprise and HPC storage systems, which indicates that the data deduplication technology can be used to effectively reduce the write traffic and storage space in such environments. However, our experimental studies reveal that applying data deduplication to primary storage systems will cause space contention in main memory and data fragmentation on disks. This is in part because applying data deduplication introduces significant index memory overhead to the existing system and in part because a file or block is split into multiple small data chunks that are often located in non-sequential locations on disks after deduplication. This fragmentation of data can cause a subsequent read operation to invoke many disk I/O requests, thus leading to performance degradation. The existing primary data deduplication schemes, such as iDedup[1], are to leverage spatial locality in that they only select the large requests to deduplicate and exclude the small requests (e.g., 4KB, 8KB or less) because the latter only account for a tiny fraction of the storage capacity requirement[2]. Moreover, these schemes tend to overlook the importance of cache management, leading them to manage the index cache and the read cache separately. However, previous workload studies on primary storage systems have revealed that small I/O requests dominate in the primary storage systems (more than 50%) and are at the root of the system performance bottleneck. Furthermore, the accesses in primary storage systems exhibit obvious I/O burstiness. The existing primary-storage data deduplication schemes fail to consider these workload characteristics in primary storage systems from the performance's perspective. We argue that, primary-storage data deduplication schemes should take the workload characteristics of primary storage into the design considerations. To address the two problems and take the primary-storage workload characteristics into considerations, we propose a Performance-Oriented I/O Deduplication approach, POD, to improving the I/O performance of primary storage systems in the Cloud. POD takes a two-pronged deduplication approach to improve primary storage systems, a request-based I/O and data deduplication scheme, called Select-Dedupe, aimed at alleviating data fragmentation and an adaptive memory management scheme, called iCache, to ease the main memory contention. More specifically, the former takes the workload characteristics of small-I/O-request domination into the design considerations. It deduplicates all the write requests if their write data is already stored sequentially on disks, including the small write requests that would otherwise be excluded from by the capacity-oriented deduplication schemes. For other write requests, Select-Dedupe does not deduplicate their redundant write data to maintain the performance of the subsequent read requests to these data. iCache takes the I/O burstiness characteristics into the design considerations. It dynamically adjusts the cache space between the index cache and the read cache according to the workload characteristics, and swaps these data between memory and backend storage devices accordingly. During the write-intensive bursty periods, iCache enlarges the index cache size and shrinks the read cache size to detect much more redundant write requests, thus improving the write performance. The read cache size is enlarged to cache more hot read data to improve the read performance during the read-intensive bursty periods. The prototype of the POD scheme is implemented as an embedded module at the block-device level with the fixed-size chunking method. Preliminary evaluations driven by the real traces conducted on our lightweight POD prototype implementation show that POD significantly outperforms iDedup in improving the performance of primary storage systems in the Cloud.
Starting Page	1
Ending Page	2
Page Count	2
File Format	PDF
ISBN	9781450324281
DOI	10.1145/2523616.2525939
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2013-10-01
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Cloud Performance Data deduplication
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in