NDLI: Flow sampling under hard resource constraints

Please wait, while we are loading the content...

Cross-architecture performance predictions for scientific applications using parameterized models

Flow classification by histograms: or how to go on safari in the internet

Two-level processor-sharing scheduling disciplines: mean delay analysis

A fast simulation framework for IEEE 802.11-operated wireless LANs

A game theoretic approach to provide incentive and service differentiation in P2P networks

On performance bounds for the integration of elastic and adaptive streaming flows

Characterization of queue fluctuations in probabilistic AQM mechanisms

A model of BGP routing for network engineering

CapProbe: a simple and accurate capacity estimation technique for wired and wireless environments

Stochastic traffic engineering for demand uncertainty and risk-aware network revenue management

Multi-dimensional storage virtualization

Structural analysis of network traffic flows

Performance analysis of LAS-based scheduling disciplines in a packet switched network

ACCEL-RATE: a faster mechanism for memory efficient per-flow traffic estimation

Failure recovery for structured P2P networks: protocol design and performance evaluation

Rate-based versus queue-based models of congestion control

The output of a cache under the independent reference model: where did the locality of reference go?

A mean-field analysis of short lived interacting TCP flows

Harpoon: a flow-level traffic generator for router and network tests

Measuring adversaries

Myths and realities: the performance impact of garbage collection

How to identify and estimate the largest traffic matrix elements in a dynamic environment

Emulating low-priority transport at the application layer: a background transfer service

VPC3: a fast and effective trace-compression algorithm

On zone-balancing of peer-to-peer networks: analysis of random node join

Uncooperative congestion control

Dynamics of hot-potato routing in IP networks

Bridging router performance and queuing theory

Spatio-temporal available bandwidth estimation with STAB

Some systems, applications and models I have known

Interposed proportional sharing for a storage service utility

Flow sampling under hard resource constraints

A resource-allocation queueing fairness measure

Data streaming algorithms for efficient and accurate estimation of flow size distribution

Performance aware tasking for environmentally powered sensor networks

Coping with network failures: routing strategies for optimal demand oblivious restoration

The impact of BGP dynamics on intra-domain traffic

Insensitive load balancing

Optimizing the quality of scalable video streams on P2P networks

Wireless data performance in multi-cell scenarios

PeerPressure for automatic troubleshooting

A quantitative analysis of partitioning in mobile ad hoc networks

Locating the tightest link of a network path

Using probabilistic reasoning to automate software tuning

Multimedia streaming via TCP: an analytic performance study

Parameter inference of queueing models for IT systems using end-to-end measurements

Performance analysis of BSTs in system software

Storage device performance prediction with CART models

Controlling the performance of 3-tiered web sites: modeling, design and implementation

Combining routing and traffic data for detection of IP forwarding anomalies

Exploring the performance benefits of end-to-end path switching

Complete or fast reference trace collection for simulating multiprogrammed workloads: choose one

Quantifying trade-offs in resource allocation for VPNs

The origins of network server latency & the myth of connection scheduling

A hybrid direct-indirect estimator of network internal delays

Multicast protocols for scalable on-demand download

Isolating the performance impacts of network interface cards through microbenchmarks

An evaluation of chord using traces of peer-to-peer file sharing

Flow sampling under hard resource constraints

Content Provider	ACM Digital Library
Author	Duffield, Nick Thorup, Mikkel Lund, Carsten
Abstract	Many network management applications use as their data traffic volumes differentiated by attributes such as IP address or port number. IP flow records are commonly collected for this purpose: these enable determination of fine-grained usage of network resources. However, the increasingly large volumes of flow statistics incur concomitant costs in the resources of the measurement infrastructure. This motivates sampling of flow records.This paper addresses sampling strategy for flow records. Recent work has shown that non-uniform sampling is necessary in order to control estimation variance arising from the observed heavy-tailed distribution of flow lengths. However, while this approach controls estimator variance, it does not place hard limits on the number of flows sampled. Such limits are often required during arbitrary downstream sampling, resampling and aggregation operations employed in analysis of the data.This paper proposes a correlated sampling strategy that is able to select an arbitrarily small number of the "best" representatives of a set of flows. We show that usage estimates arising from such selection are unbiased, and show how to estimate their variance, both offline for modeling purposes, and online during the sampling itself. The selection algorithm can be implemented in a queue-like data structure in which memory usage is uniformly bounded during measurement. Finally, we compare the complexity and performance of our scheme with other potential approaches.
Starting Page	85
Ending Page	96
Page Count	12
File Format	PDF
ISBN	1581138733
DOI	10.1145/1005686.1005699
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2004-06-10
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Ip flows Variance reduction Sampling
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Flow sampling under hard resource constraints

Flow Sampling Under Hard Resource Constraints

Predicting resource usage and estimation accuracy in an IP flow measurement collection infrastructure

Fair sampling across network flow measurements

Estimating flow distributions from sampled flow statistics

Reduction of traffic sampling impact on anomaly detection

A Resource-Efficient System for Measuring Flow Length Distribution

Learn more, sample less: control of volume and variance in network measurement

Sampling the repairs of functional dependency violations under hard constraints.

Flow sampling under hard resource constraints

Similar Documents

Flow sampling under hard resource constraints

Flow Sampling Under Hard Resource Constraints

Predicting resource usage and estimation accuracy in an IP flow measurement collection infrastructure

Fair sampling across network flow measurements

Estimating flow distributions from sampled flow statistics

Reduction of traffic sampling impact on anomaly detection

A Resource-Efficient System for Measuring Flow Length Distribution

Learn more, sample less: control of volume and variance in network measurement

Sampling the repairs of functional dependency violations under hard constraints.

Flow sampling under hard resource constraints