NDLI: Predicting SQL Query Execution Time for Large Data Volume

Please wait, while we are loading the content...

Web Data Management in the RDF Age: Keynote talk abstract

A Deterministic Model for P2P Deductive Databases

A New XML Schema based on Type Dependencies

Panel: The State of Data: Invited Paper from panelists

A new scalable aggregation scheme for fuzzy clustering Taking unstructured textual resources as a case

The Twitter Bullishness Index: A Social Media Analytics Indicator for the Stock Market

Computing Marginals Using MapReduce: Keynote talk paper

Efficient Approximate OLAP Querying Over Time Series

Clickstream Prediction Using Sequential Stream Mining Techniques with Markov Chains

A Big Data Approach For Querying Data in EHR Systems

TPRED: a Spatio-Temporal Location Predictor Framework

Ontology-Based Data Integration System for Conservation-Restoration Data (OBDIS-CR)

A Two-Phase MapReduce Algorithm for Scalable Preference Queries over High-Dimensional Data

A Case Study of the Aggregation Query Model in Read-Mostly NoSQL Document Stores

On the Discovery of Relaxed Functional Dependencies

Performance Evaluation of NoSQL Multi-Model Data Stores in Polyglot Persistence Applications

An Innovative Framework for Effectively and Efficiently Supporting Big Data Analytics over Geo-Located Mobile Social Media

On computing temporal functions for a time-dependent networks using trajectory data

The Influence of Crowd Type and Task Complexity on Crowdsourced Work Quality

How, Who and When: Enhancing Business Process Warehouses By Graph Based Queries

Optimizing Druid with Roaring bitmaps

Valet SADI: Provisioning SADI Web Services for Semantic Querying of Relational Databases

Data on the move and Issues of Privacy and security: Dangers of the web

A New Approach to Preserving Data Confidentiality in the Cloud

Content-preserving Text Watermarking through Unicode Homoglyph Substitution

JARS: Join-Aware Distributed RDF Storage

Mintra: Mining anonymized trajectories with annotations

XML Type Dependency and XML Type Normal Form

Lazy Evaluation for Concurrent OLTP and Bulk Transactions

Secure and Efficient Multiparty Computation on Genomic Data

Using Histograms for Skyline Size Estimation

Introducing a New Arabic Endpoint for DBpedia Internationalization Project

Improving GraphChi for Large Graph Processing: Fast Radix Sort in Pre-Processing

A Large-Scale Study of Online Shopping Behavior

DynamicDFEP: A Distributed Edge Partitioning Approach for Large Dynamic Graphs

CUTiS: optimized online ClUstering of Trajectory data Stream

Software Analytics in Practice: A Defect Prediction Model Using Code Smells

The Butterfly: An Intelligent Framework for Violation Prediction within Business Processes

Enhanced variational approach by Fast-Marching to recover a 3D shape from shading

Mobile Web Application Generation Features for SuperSQL

The Hilbert PDC-tree: A High-Velocity Structure for Many-Dimensional Data

Hierarchical Crowd Curation of Very Large and Predicted Protein-Protein Interaction Data

Enriching Mobility Data with Linked Open Data

Enhancing applications with filtering of XML message streams

Comparative Mining of B2C Web Sites by Discovering Web Database Schemas

SCI-F: Social-Corporate Data Integration Framework

A Scientist's Impact over Time: The Predictive Power of Clustering with Peers

Modern Web Page Generation by Using Exclusively SuperSQL

Optimization of a Class of Temporal Queries

Extension of Semantic Generalization for Considering a Class-Lattice

Visual Spatial-OLAP for Vehicle Recorder Data on Micro-sized Electric Vehicles

The Extraction from News Stories a Causal Topic Centred Bayesian Graph for Sugarcane

Conceptual and Database Modelling of Graph Databases

Predicting SQL Query Execution Time for Large Data Volume

Provenance Support for Biomedical Big Data Analytics

Predicting SQL Query Execution Time for Large Data Volume

Content Provider	ACM Digital Library
Author	Nambiar, Manoj Singhal, Rekha
Abstract	In a production system, increase in data size will increase the execution time of the application's SQL queries and degrade its performance. Tuning SQL queries in production requires additional efforts and cost. Time constraints during application development do not permit testing SQL queries with high data volumes. Having the capability to predict SQL query execution time for large data volumes can alert the developers to tune queries or database design upfront, in such scenarios. Application developers may use 'cost' of SQL query as given by optimizer based relational databases to estimate the SQL query execution time for large data sizes. However, the 'cost' based models may lead to large estimation errors as discussed in this paper. We have presented a modular approach of estimating SQL query execution time for high data volumes using measurements at low data volume. A compound SQL query execution plan is mapped to a sequential execution of a set of elementary steps. The execution time of a SQL query in isolation is predicted as summation of estimated execution time of all its elementary steps. We have built analytical models for estimating execution time of different IO access, DB cache access and SQL operators as function of data size for each such step. The proposed model dynamically adapts itself to the structure of the query execution plans and characteristics of the underlying hardware. We have evaluated the model by generating synthetic queries for all combinations of elementary steps for a range of data sizes. The model has also been validated with TPC-H benchmarks and three real life applications. The proposed model shows an average prediction error to be within 10%.
Starting Page	378
Ending Page	385
Page Count	8
File Format	PDF
ISBN	9781450341189
DOI	10.1145/2938503.2938552
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-07-11
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Prediction Data volume Big data Performance Sql
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A Framework for Predicting Query Response Time

SQL Query Volume Performance Estimation Tool

Reducing structured Big data benchmark cycle time using query performance prediction model

Sphinx: distributed execution of interactive SQL queries on big spatial data

Proactive performance testing using SQL performance assurance services (SQL-PASS)

Big data analysis and query optimization improve HadoopDB performance

Efficient processing of data warehousing queries in a split execution environment.

An Optimization Method of Data Communication and Control for Parallel Execution of SQL Queries (1993)

SQL: From Traditional Databases to Big Data

Predicting SQL Query Execution Time for Large Data Volume

Similar Documents

A Framework for Predicting Query Response Time

SQL Query Volume Performance Estimation Tool

Reducing structured Big data benchmark cycle time using query performance prediction model

Sphinx: distributed execution of interactive SQL queries on big spatial data

Proactive performance testing using SQL performance assurance services (SQL-PASS)

Big data analysis and query optimization improve HadoopDB performance

Efficient processing of data warehousing queries in a split execution environment.

An Optimization Method of Data Communication and Control for Parallel Execution of SQL Queries (1993)

SQL: From Traditional Databases to Big Data

Predicting SQL Query Execution Time for Large Data Volume