NDLI: StreamWeb: Real-Time Web Monitoring with Stream Computing

Content Provider	IEEE Xplore Digital Library
Author	Suzumura, T. Oiki, T.
Copyright Year	2011
Abstract	A new trend involves Web services such as Twitter beginning to publish streaming Web APIs that enable partners and end users to retrieve streaming data. By combining such push-based Web services and existing pull-based Web services, it is now possible for us to understand the current status or trends of the world in a more real-time way, such as real-time tracking of infectious disease, real-time crime prediction, or real-time marketing, and so various innovative business services are possible. For a system architecture to implement such services, the services are normally built from the scratch, and the performance and scalability depend upon the engineers' skills. In this paper we propose a real-time Web monitoring system called gStreamWebh on top of a stream computing system called System S developed by IBM Research. The Stream Web system allows developers to easily describe their analytical algorithms for a variety of kinds of Web streaming data without worrying about the performance and scalability, and provides real-time and scalable Web monitoring for massive amounts of data. As an experimental proof-of-concept application, we built an application that monitors a list of keywords in the Twitter streaming data, and that displays any messages including the specified keywords onto a map of the physical location (from Google) where the message was posted. Our system can handle nearly 30 thousand Twitter messages per second on a system with 8 computing nodes. This prototype application confirms that we can build real-time Web monitoring systems while satisfying the needs for high software productivity and for system scalability.
Starting Page	620
Ending Page	627
File Size	1184872
Page Count	8
File Format	PDF
ISBN	9781457708428
DOI	10.1109/ICWS.2011.16
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2011-07-04
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Real time systems Twitter Web services Monitoring Engines Data visualization XML real-time stream computing system s web monitoring
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in