NDLI: FlatFIT: Accelerated Incremental Sliding-Window Aggregation For Real-Time Analytics

Please wait, while we are loading the content...

Managing Sensor Data Streams: Lessons Learned from the WeBike Project

Measuring Fairness in Ranked Outputs

SGVCut: A Vertex-Cut Partitioning Tool for Random Walks-based Computations over Social Network graphs

Incremental Discovery of Inclusion Dependencies

How the Passengers Flow in Complex Metro Networks?

VISOR: Visualizing Summaries of Ordered Data

Generating What-If Scenarios for Time Series Data

DualDB: An Efficient LSM-based Publish/Subscribe Storage System

Skluma: A Statistical Learning Pipeline for Taming Unkempt Data Repositories

Computing Influence of a Product through Uncertain Reverse Skyline

DistriPlan: An Optimized Join Execution Framework for Geo-Distributed Scientific Data

DataSynthesizer: Privacy-Preserving Synthetic Datasets

FlatFIT: Accelerated Incremental Sliding-Window Aggregation For Real-Time Analytics

Fides: Towards a Platform for Responsible Data Science

A Benchmark for Betweenness Centrality Approximation Algorithms on Large Graphs

Challenges of Differentially Private Release of Data Under an Open-world Assumption

Incremental Temporal Pattern Mining Using Efficient Batch-Free Stream Clustering

Mining Persistent and Discriminative Communities in Graph Ensembles

Dot-Product Join: Scalable In-Database Linear Algebra for Big Model Analytics

High-Definition Digital Elevation Model System Vision Paper

On-line Versioned Schema Inference for Large Semantic Web Data Sources

Discovering Partial Periodic Itemsets in Temporal Databases

Bi-Level Online Aggregation on Raw Data

Detecting Global Hyperparaboloid Correlated Clusters Based on Hough Transform

Data Series Similarity Using Correlation-Aware Measures

Tiling Strategies for Distributed Point Cloud Databases

Edge Labeling Schemes for Graph Data

Computing Isochrones in Multimodal Spatial Networks using Tile Regions

Class-based Conditional MaxRS Query in Spatial Data Streams

Location and Processing Aware Datacube Caching

Active Learning with Density-Initialized Decision Tree for Record Matching

MELT: Mapreduce-based Efficient Large-scale Trajectory Anonymization

BLOCK: Efficient Execution of Spatial Range Queries in Main-Memory

PLI: Augmenting Live Databases with Custom Clustered Indexes

Multi-Hypothesis CSV Parsing

Improving Statistical Similarity Based Data Reduction for Non-Stationary Data

Fast Equi-Join Algorithms on GPUs: Design and Implementation

Dynamic Group Trip Planning Queries in Spatial Databases

Query Suggestion to allow Intuitive Interactive Search in Multidimensional Time Series

SHRec: Scalable Holistic Recommendation

A Unified Correlation-based Approach to Sampling Over Joins

Probabilistic k-Nearest Neighbor Monitoring of Moving Gaussians

FlatFIT: Accelerated Incremental Sliding-Window Aggregation For Real-Time Analytics

Content Provider	ACM Digital Library
Author	Chrysanthis, Panos K. Labrinidis, Alexandros Shein, Anatoli U.
Abstract	Data stream processing is becoming essential in most current advanced scientific or business applications as data production rates are increasing. Different companies compete to efficiently ingest high velocity data and apply some form of computation in order to make better business decisions. In order to successfully compete in this environment, companies are focusing on the most recent data within a count or time-based window by continuously executing aggregate queries on it. Incremental sliding-window computation is commonly used to avoid the performance implications of re-evaluating the aggregate value of the window from scratch on every update. The state-of-the-art FlatFAT technique executes ACQs with high efficiency but it does not scale well with the increasing workloads. In this paper we propose a novel algorithm, FlatFIT, that accelerates such calculations by intelligently maintaining index structures, leading to higher reuse of intermediate calculations and thus exceptional scalability in systems with heavy workloads. Our theoretical analysis shows that FlatFIT is superior in both time and space complexities compared to FlatFAT, while maintaining the same query generality. Given a window of size n, FlatFIT achieves constant algorithmic complexity compared to O(log(n)) complexity of FlatFAT. We experimentally show that FlatFIT achieves up to a 17x throughput improvement over FlatFAT for the same input workload while using less memory.
Starting Page	1
Ending Page	12
Page Count	12
File Format	PDF
ISBN	9781450352826
DOI	10.1145/3085504.3085509
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2017-06-27
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Sliding-window processing Flatfit Aggregate continuous query
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in