NDLI: StreamCloud: A Large Scale Data Streaming System

Content Provider	IEEE Xplore Digital Library
Author	Gulisano, V. Jimenez-Peris, R. Patino-Martinez, M. Valduriez, P.
Copyright Year	2010
Abstract	Data streaming has become an important paradigm for the real-time processing of continuous data flows in domains such as finance, telecommunications, networking, Some applications in these domains require to process massive data flows that current technology is unable to manage, that is, streams that, even for a single query operator, require the capacity of potentially many machines. Research efforts on data streaming have mainly focused on scaling in the number of queries or query operators, but overlooked the scalability issue with respect to the stream volume. In this paper, we present StreamCloud a large scale data streaming system for processing large data stream volumes. We focus on how to parallelize continuous queries to obtain a highly scalable data streaming infrastructure. StreamCloud goes beyond the state of the art by using a novel parallelization technique that splits queries into subqueries that are allocated to independent sets of nodes in a way that minimizes the distribution overhead. StreamCloud is implemented as a middleware and is highly independent of the underlying data streaming engine. We explore and evaluate different strategies to parallelize data streaming and tackle with the main bottlenecks and overheads to achieve scalability. The paper presents the system design, implementation and a thorough evaluation of the scalability of the fully implemented system.
Starting Page	126
Ending Page	137
File Size	575268
Page Count	12
File Format	PDF
ISBN	9781424472611
ISSN	10636927
e-ISBN	9781424472628
DOI	10.1109/ICDCS.2010.72
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-06-21
Publisher Place	Italy
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Large-scale systems Scalability Engines Telephony Distributed computing Real time systems Finance Technology management Middleware System analysis and design
Content Type	Text
Resource Type	Article
Subject	Computer Networks and Communications Hardware and Architecture Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

2010 international conference on distributed computing systems streamcloud: a large scale data streaming system ⋆.

Ultra-scalable architectures for Telecommunications and Web 2.0 services

A microkernel middleware architecture for distributed embedded real-time systems

Distributed Slicing in Dynamic Systems

Toward nex-generation middleware?

Towards a Scalable Real-Time Cyberinfrastructure for Online Computer Games

Lightweight Morphing Support for Evolving Middleware Data Exchanges in Distributed Applications

StreamCloud: An Elastic and Scalable Data Streaming System

Uncertainty Analysis of Middleware Services for Streaming Smart Grid Applications

StreamCloud: A Large Scale Data Streaming System

Similar Documents

2010 international conference on distributed computing systems streamcloud: a large scale data streaming system ⋆.

Ultra-scalable architectures for Telecommunications and Web 2.0 services

A microkernel middleware architecture for distributed embedded real-time systems

Distributed Slicing in Dynamic Systems

Toward nex-generation middleware?

Towards a Scalable Real-Time Cyberinfrastructure for Online Computer Games

Lightweight Morphing Support for Evolving Middleware Data Exchanges in Distributed Applications

StreamCloud: An Elastic and Scalable Data Streaming System

Uncertainty Analysis of Middleware Services for Streaming Smart Grid Applications

StreamCloud: A Large Scale Data Streaming System