NDLI: Counting the number of active flows on a high speed link

Please wait, while we are loading the content...

Counting the number of active flows on a high speed link

Content Provider	ACM Digital Library
Author	Estan, Cristian Varghese, George Fisk, Mike
Abstract	Internet links operate at high speeds, and past trends predict that these speeds will continue to increase rapidly. Routers and Intrusion Detection Systems that operate at up to OC-768 speeds (40 Gigabits/second) are currently being developed. In this paper we address a basic function common to several security and measurement applications running at line speeds: counting the number of distinct header patterns (flows) seen on a high speed link in a specified time period. For example one can detect port scans by counting the number of connections opened by the suspected port scanner and one can ascertain that a denial of service attack is in progress by counting the number of distinct (fake) source addresses that send packets to the suspected victim. We provide algorithms solving the flow counting problem using extremely small amounts of memory. This can translate into savings of scarce fast memory (SRAM) for hardware implementations. It can also help systems that use cheaper DRAM to allow them to scale to larger instances of the problem. The reduction in memory is particularly important for network security applications such as detecting port scans and DoS attacks that need to run a separate instance of the algorithm for each suspected attacker or victim. Our algorithms can be implemented in hardware at wire speeds (8 nsec per packet for OC-768) using simple CRC based hash functions, multiplexers and SRAM. They access at most one or two memory locations per packet. We expose the behavior of simple building blocks and provide a family of customizable counting algorithms that can be adapted to the requirements of various applications. This allows our algorithms to use less memory than the best known counting algorithm, probabilistic counting, to provide the same accuracy.• Virtual bitmap is well suited for triggers (which need only be accurate around a threshold value) such as detecting DoS attacks, and uses 292 bytes to achieve an error of 2.619% compared to 5,200 bytes for probabilistic counting.• Our "no assumptions" counting algorithm, multiresolution bitmap algorithms uses only 2143 bytes to count up to 100 million flows with an average error of 3%. Its accuracy is slightly better when the number of flows is small, while the accuracy of probabilistic counting using the same memory is much worse.• Our adaptive bitmap that exploits stationarity in the number of flows can count the number of distinct flows on a link that contains anywhere from 0 to 100 million flows with an average error of less than 1 % using only 2 Kbytes of memory. Probabilistic counting needs eight times more memory to achieve the same accuracy.• We used our triggered bitmap to replace the port scan detection component of the popular intrusion detection system Snort. Doing so reduced the memory usage from 89 Mbytes to 6.4 Mbytes (with an average error of 13.5%) as measured on a 10 minute trace. Probabilistic counting uses 23 Mbytes.This work was made possible by a grant from NIST for the Sensilla Project.
Starting Page	10
Ending Page	10
Page Count	1
File Format	PDF
ISSN	01464833
DOI	10.1145/571697.571699
Journal	ACM SIGCOMM Computer Communication Review (CCRV)
Volume Number	32
Issue Number	3
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	1993-07-01
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Content Type	Text
Resource Type	Article
Subject	Computer Networks and Communications Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Bitmap algorithms for counting active flows on high-speed links

Counting the number of active flows on a high speed link

Bitmap Algorithms for Counting Active Flows on High-Speed Links

Measuring the performance of communication middleware on high-speed networks

Data link control emulation: rapid prototyping for high-speed networks

Review of the Journal of High Speed Networks

Report on the Workshop on Quality of Service Issues in High Speed Networks

Experiences with a high-speed network adaptor:a software perspective

Time-shift scheduling-fair scheduling of flows in high-speed networks

Counting the number of active flows on a high speed link

Similar Documents

Bitmap algorithms for counting active flows on high-speed links

Counting the number of active flows on a high speed link

Bitmap Algorithms for Counting Active Flows on High-Speed Links

Measuring the performance of communication middleware on high-speed networks

Data link control emulation: rapid prototyping for high-speed networks

Review of the Journal of High Speed Networks

Report on the Workshop on Quality of Service Issues in High Speed Networks

Experiences with a high-speed network adaptor:a software perspective

Time-shift scheduling-fair scheduling of flows in high-speed networks

Counting the number of active flows on a high speed link