NDLI: Query-Aware Partitioning for Monitoring Massive Network Data Streams

Content Provider	IEEE Xplore Digital Library
Author	Johnson, T. Muthukrishnan, S. Shkapenyuk, V. Spatscheck, O.
Copyright Year	2008
Description	Author affiliation: AT&T Labs. - Res., Florham Park, NJ (Johnson, T.)
Abstract	Data stream management systems (DSMS) are gaining acceptance for applications that need to process very large volumes of data in real time. The load generated by such applications frequently exceeds by far the computation capabilities of a single centralized server. In particular, a single-server instance of our DSMS, Gigascope, cannot keep up with the processing demands of the new OC-786 networks, which can generate more than 100 million packets per second. In this paper, we explore a mechanism for the distributed processing of very high speed data streams. Existing distributed DSMSs employ two mechanisms for distributing the load across the participating machines: partitioning of the query execution plans and partitioning of the input data stream in a query-independent fashion. However, for a large class of queries, both approaches fail to reduce the load as compared to centralized system, and can even lead to an increase in the load. In this paper we present an alternative approach - query-aware data stream partitioning that allows for more efficient scaling. We have developed methods for analyzing any given query node to determine a partition strategy, reconcile potentially conflicting requirements that different queries in a query set place on partitioning, and to choose an optimal partitioning which minimizes overall communication costs..
Starting Page	1528
Ending Page	1530
File Size	886222
Page Count	3
File Format	PDF
ISBN	9781424418367
DOI	10.1109/ICDE.2008.4497612
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2008-04-07
Publisher Place	Mexico
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Condition monitoring Computer network management Network servers Computer networks Telecommunication traffic Web server Real time systems Distributed processing Cost function Feeds
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Query-Aware Sampling for Data Streams

Query-aware partitioning for monitoring massive network data streams

A unified framework for monitoring data streams in real time

A Web-based management system for network monitoring

Query Optimization over Distributed Data Stream

Speculative data dissemination and service to reduce server load, network traffic and service time in distributed information systems

Flamingo: Visualizing Internet Traffic

Adaptive distributed caching

Resource-aware real-time CORBA in multi-server distributed environment

Query-Aware Partitioning for Monitoring Massive Network Data Streams

Similar Documents

Query-Aware Sampling for Data Streams

Query-aware partitioning for monitoring massive network data streams

A unified framework for monitoring data streams in real time

A Web-based management system for network monitoring

Query Optimization over Distributed Data Stream

Speculative data dissemination and service to reduce server load, network traffic and service time in distributed information systems

Flamingo: Visualizing Internet Traffic

Adaptive distributed caching

Resource-aware real-time CORBA in multi-server distributed environment

Query-Aware Partitioning for Monitoring Massive Network Data Streams