NDLI: Robust traffic matrix estimation with imperfect information: making use of multiple data sources

Please wait, while we are loading the content...

Stardust: tracking activity in a distributed storage system

Maximizing throughput in wireless networks via gossiping

GPS scheduling: selection of optimal weights and comparison with strict priorities

NetQuest: a flexible framework for large-scale network measurement

An efficient algorithm for the exact analysis of multiclass queueing networks with large population sizes

Automatic logging of operating system effects to guide application-level architecture simulation

Confidence intervals for priority sampling

Robust network connectivity: when it's the big picture that matters

Fluid modeling of pollution proliferation in P2P networks

QuickProbe: available bandwidth estimation in two roundtrips

Performance and reliability: the ubiquitous challenge

Exploiting redundancy to conserve energy in storage systems

Determining the end-to-end throughput capacity in multi-hop networks: methodology and applications

The impact of reneging in processor sharing queues

Robust traffic matrix estimation with imperfect information: making use of multiple data sources

Transient analysis of tree-Like processes and its application to random access systems

An analytical model for cache replacement policy performance

Finding probably better system configurations quickly

Packet classifiers in ternary CAMs can be smaller

Fast statistical spam filter by approximate classifications

Flexible time-windows for advance reservation in LambdaGrids

Impacts of channel variability on link-level throughput in wireless networks

Tail asymptotics for policies favoring short jobs in a many-flows regime

Data streaming algorithms for estimating entropy of network traffic

Bounding stationary results of Tandem networks with MAP input and PH service time distributions

Understanding the management of client perceived response time

The Erlang model with non-poisson call arrivals

Design of a novel statistics counter architecture with optimal space and time efficiency

Analyzing persistent state interactions to improve state management

Partially overlapped channels not considered harmful

On light and heavy traffic approximations of balanced fairness

On suitability of Euclidean embedding of internet hosts

Fundamental characteristics of queues with fluctuating load

On the way to a distributed systems calculus: an end-to-end network calculus with data scaling

Delay-optimal scheduling in bandwidth-sharing networks

Throughput performance of popular JMS servers

Optimizing the HPCC randomaccess benchmark on blue Gene/L Supercomputer

Peer-to-peer streaming of stored media: the indirect approach

Analysis of a new intra-disk redundancy scheme for high-reliability RAID storage systems in the presence of unrecoverable errors

Applying architectural vulnerability Analysis to hard faults in the microprocessor

A multicommodity flow model for distributed stream processing

A characterization of a java-based commercial workload on a high-end enterprise server

Optimizing transmission rate in wireless channels using adaptive probes

Consolidating clients on back-end servers with co-location and frequency control

Congestion responsiveness of internet traffic: (a fresh look at an old problem)

Towards deterministic network diagnosis

Modeling adoptability of secure BGP protocols

Robust traffic matrix estimation with imperfect information: making use of multiple data sources

Content Provider	ACM Digital Library
Author	Zhao, Qi Wang, Jia Xu, Jun Ge, Zihui
Abstract	Estimation of traffic matrices, which provide critical input for network capacity planning and traffic engineering, has recently been recognized as an important research problem. Most of the previous approaches infer traffic matrix from either SNMP link loads or sampled NetFlow records. In this work, we design novel inference techniques that, by statistically correlating SNMP link loads and sampled NetFlow records, allow for much more accurate estimation of traffic matrices than obtainable from either information source alone, even when sampled NetFlow records are available at only a subset of ingress. Our techniques are practically important and useful since both SNMP and NetFlow are now widely supported by vendors and deployed in most of the operational IP networks. More importantly, this research leads us to a new insight that SNMP link loads and sampled NetFlow records can serve as "error correction codes" to each other. This insight helps us to solve a challenging open problem in traffic matrix estimation, "How to deal with dirty data (SNMP and NetFlow measurement errors due to hardware/software/transmission problems)?" We design techniques that, by comparing notes between the above two information sources, identify and remove dirty data, and therefore allow for accurate estimation of the traffic matrices with the cleaned dat.We conducted experiments on real measurement data obtained from a large tier-1 ISP backbone network. We show that, when full deployment of NetFlow is not available, our algorithm can improve estimation accuracy significantly even with a small fraction of NetFlow data. More importantly, we show that dirty data can contaminate a traffic matrix, and identifying and removing them can reduce errors in traffic matrix estimation by up to an order of magnitude. Routing changes is another a key factor that affects estimation accuracy. We show that using them as the a priori, the traffic matrices can be estimated much more accurately than those omitting the routing change. To the best of our knowledge, this work is the first to offer a comprehensive solution which fully takes advantage of using multiple readily available data sources. Our results provide valuable insights on the effectiveness of combining flow measurement and link load measurement.
Starting Page	133
Ending Page	144
Page Count	12
File Format	PDF
ISBN	1595933190
DOI	10.1145/1140277.1140294
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2006-06-26
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Network measurement Statistical inference Traffic matrix
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Robust Traffic Matrix Estimation with Imperfect Information: Making Use of Multiple Data Sources (2006)

Robust traffic matrix estimation with imperfect information:making use of multiple data sources

Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices

Traffic matrices: balancing measurements, inference and modeling (2005)

An Improved Link Weight Change Algorithm for Traffic Matrix Estimation

Traffic matrices: balancing measurements, inference and modeling (2005)

Traffic matrix estimation: existing techniques and new directions

Data streaming algorithms for efficient and accurate estimation of flow size distribution

Traffic matrix inference in ip networks.

Robust traffic matrix estimation with imperfect information: making use of multiple data sources

Similar Documents

Robust Traffic Matrix Estimation with Imperfect Information: Making Use of Multiple Data Sources (2006)

Robust traffic matrix estimation with imperfect information:making use of multiple data sources

Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices

Traffic matrices: balancing measurements, inference and modeling (2005)

An Improved Link Weight Change Algorithm for Traffic Matrix Estimation

Traffic matrices: balancing measurements, inference and modeling (2005)

Traffic matrix estimation: existing techniques and new directions

Data streaming algorithms for efficient and accurate estimation of flow size distribution

Traffic matrix inference in ip networks.

Robust traffic matrix estimation with imperfect information: making use of multiple data sources