NDLI: MiCA: Real-Time Mixed Compression Scheme for Large-Scale Distributed Monitoring

Content Provider	IEEE Xplore Digital Library
Author	Bo Wang Ying Song Yuzhong Sun Jun Liu
Copyright Year	2014
Description	Author affiliation: State Key Lab. of Comput. Archit., Inst. of Comput. Technol., Beijing, China (Ying Song; Yuzhong Sun) \|\| Sch. of Electron. & Inf. Eng., Xi'an Jiaotong Univ., Xi'an, China (Bo Wang; Jun Liu)
Abstract	Real-time monitoring, providing the real-time status information of servers, is indispensable for the management of distributed systems, e.g. failure detection and resource scheduling. The scalability of fine-grained monitoring faces more and more severe challenges with scaling up distributed systems. The real-time compression which suppresses remote information update to reduce continuous monitoring cost is a promising approach to address the scalability problem. In this paper, we present the Linear Compression Algorithm (LCA) which is the application of the linear filter to real-time monitoring. To our best knowledge, existing work and LCA only explores the correlations of values of each single metric at various times. We present a novel lightweight REal-time Compression Algorithm (ReCA) which employs discovery methods of the correlation among metrics to suppress remote information update in distributed monitoring. The compression algorithms mentioned above have limited compression power because they only explore either the correlations of values of each single metric at various times or that among metrics. Therefore, we propose the Mixed Compression Algorithm (MiCA) which explores both of the correlations to achieve higher compression ratio. We implement our algorithms and an existing compression algorithm denoted by CCA in a distributed monitoring system Ganglia and conduct extensive experiments. The experimental results show that LCA and ReCA have comparable compression ratios with CCA, that MiCA achieves up to 38.2%, 27% and 44.5% higher compression ratios than CCA, LCA and ReCA with negligible overhead, respectively, and that LCA, and ReCA can both increase the scalability of Ganglia about 1.5 times and MiCA can increase about 2.33 times under a mixed-load circumstance.
Starting Page	441
Ending Page	450
File Size	703066
Page Count	10
File Format	PDF
ISBN	9781479956180
ISSN	01903918
DOI	10.1109/ICPP.2014.53
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-09-09
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Measurement Correlation Monitoring Compression algorithms Real-time systems Peer-to-peer computing Scalability real-time data compression distributed system monitoring
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Aggregation of Real-Time System Monitoring Data for Analyzing Large-Scale Parallel and Distributed Computing Environments

Dynamic resource discovery for applications survivability in distributed real-time systems

Distributed spanning tree algorithms for large scale traversals

ServiceDDS: A Framework for Real-Time P2P Systems Integration

EMMON: A WSN System Architecture for Large Scale and Dense Real-Time Embedded Monitoring

RT-P2P: A Scalable Real-Time Peer-to-Peer System with Probabilistic Timing Assurances

Real-time adaptive algorithm for resource monitoring

Scalable Multi-purpose Network Representation for Large Scale Distributed System Simulation

Randomized protocols for duplicate elimination in peer-to-peer storage systems

MiCA: Real-Time Mixed Compression Scheme for Large-Scale Distributed Monitoring

Similar Documents

Aggregation of Real-Time System Monitoring Data for Analyzing Large-Scale Parallel and Distributed Computing Environments

Dynamic resource discovery for applications survivability in distributed real-time systems

Distributed spanning tree algorithms for large scale traversals

ServiceDDS: A Framework for Real-Time P2P Systems Integration

EMMON: A WSN System Architecture for Large Scale and Dense Real-Time Embedded Monitoring

RT-P2P: A Scalable Real-Time Peer-to-Peer System with Probabilistic Timing Assurances

Real-time adaptive algorithm for resource monitoring

Scalable Multi-purpose Network Representation for Large Scale Distributed System Simulation

Randomized protocols for duplicate elimination in peer-to-peer storage systems

MiCA: Real-Time Mixed Compression Scheme for Large-Scale Distributed Monitoring