NDLI: Estimating Top N Hosts in Cardinality Using Small Memory Resources

Content Provider	IEEE Xplore Digital Library
Author	Ishibashi, K. Mori, T. Kawahara, R. Hirokawa, Y. Kobayashi, A. Yamamoto, K. Sakamoto, H.
Copyright Year	2006
Description	Author affiliation: NTT Corporation (Ishibashi, K.)
Abstract	We propose a method to find N hosts that have the N highest cardinalities, where cardinality is the number of distinct items such as the number of flows, ports, or peer hosts. The method also estimates their cardinalities. While existing algorithms to find the top N frequent items can be directly applied to find N hosts that send the N largest numbers of packets through packet data stream, finding hosts that have the N highest cardinalities requires tables of previously seen items for each host to check whether an item of an arrival packet is new, which requires a lot of memory. Even if we use the existing cardinality estimation methods, we still need to have cardinality information about each host. In this paper, we use the property of cardinality estimation, in which the cardinality of intersections of multiple data sets can be estimated with cardinality information of each data set. Using the property, we propose an algorithm that does not need to maintain tables for each host, but only for partitioned addresses of a host and estimate the cardinality of a host as the intersection of cardinalities of partitioned addresses. We also propose a method to find top N hosts in cardinalities which is to be monitored to detect anomalous behavior in networks. We evaluate our algorithm through actual backbone traffic data. While the estimation accuracy of our scheme degrades for small cardinalities, as for the top 100 hosts, the accuracy of our algorithm with 4, 096 tables is almost the same as having tables of every hosts.
Starting Page	29
Ending Page	29
File Size	1314701
Page Count	1
File Format	PDF
ISBN	0769525717
DOI	10.1109/ICDEW.2006.56
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2006-04-03
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Degradation Network servers Data security Laboratories Spine Telecommunication traffic Hardware Communication system traffic control Partitioning algorithms Monitoring
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Fast traffic anomalies detection using SNMP MIB correlation analysis

Dynamic networking refinement in distributed systems

Anomaly detection for network servers using digital signature of network segment

A hybrid genetic algorithm for constrained hardware-software partitioning

Application of Clustering Algorithms in Ip Traffic Classification

Modeling and Simulation of Haipe

Volunteer-based distributed traffic data collection system

Real-time simulation with traffic monitoring tool

AKQUI: a network traffic measurement tool

Estimating Top N Hosts in Cardinality Using Small Memory Resources

Similar Documents

Fast traffic anomalies detection using SNMP MIB correlation analysis

Dynamic networking refinement in distributed systems

Anomaly detection for network servers using digital signature of network segment

A hybrid genetic algorithm for constrained hardware-software partitioning

Application of Clustering Algorithms in Ip Traffic Classification

Modeling and Simulation of Haipe

Volunteer-based distributed traffic data collection system

Real-time simulation with traffic monitoring tool

AKQUI: a network traffic measurement tool

Estimating Top N Hosts in Cardinality Using Small Memory Resources