NDLI: SAGA: array storage as a DB with support for structural aggregations

Please wait, while we are loading the content...

Maintaining a microbial genome & metagenome data analysis system in an academic setting

Offline cleaning of RFID trajectory data

SAGA: array storage as a DB with support for structural aggregations

Data perturbation for outlier detection ensembles

(k, d)-core anonymity: structural anonymization of massive networks

SensorBench: benchmarking approaches to processing wireless sensor network data

PStore: an efficient storage framework for managing scientific data

Geometric graph matching and similarity: a probabilistic approach

A case study in optimizing continuous queries using the magic update technique

DistillFlow: removing redundancy in scientific workflows

Efficient classification of billions of points into complex geographic regions using hierarchical triangular mesh

Skew-resistant parallel in-memory spatial join

Extending the SQL array concept to support scientific analytics

A subspace filter supporting the discovery of small clusters in very noisy datasets

Matching dominance: capture the semantics of dominance for multi-dimensional uncertain objects

Managing evolving shapes in sensor networks

Inverse predictions on continuous models in scientific databases

Schema matching over relations, attributes, and data values

Node classification in uncertain graphs

MR-microT: a MapReduce-based MicroRNA target prediction method

Mining statistically sound co-location patterns at multiple distances

SLACID - sparse linear algebra in a column-oriented in-memory database system

DivIDE: efficient diversification for interactive data exploration

A provable algorithmic approach to product selection problems for market entry and sustainability

Proactive adaptations in sensor network query processing

Helping scientists reconnect their datasets

Point cloud databases

Integrating fault-tolerance and elasticity in a distributed data stream processing system

Integrating non-spatial preferences into spatial location queries

Efficient data management and statistics with zero-copy integration

Local context selection for outlier ranking in graphs with multiple numeric node attributes

Distributed data placement to minimize communication costs via graph partitioning

Boosting the basic counting on distributed streams

Toward efficient and reliable genome analysis using main-memory database systems

Simulation workflow design tailor-made for scientists

Efficient processing of exploratory top-k joins

A system for efficient and simultaneous processing of moving K nearest neighbor and spatial keyword queries

A study of partitioning and parallel UDF execution with the SAP HANA database

Subspace anytime stream clustering

Efficient temporal shortest path queries on evolving social graphs

Data movement in hybrid analytic systems: a case for automation

Protection of sensitive trajectory datasets through spatial and temporal exchange

New approaches to storing and manipulating multi-dimensional sparse arrays

Exploring subspace clustering for recommendations

Data patterns to alleviate the design of scientific workflows exemplified by a bone simulation

Communication-efficient preference top-k monitoring queries via subscriptions

On efficiently generating realistic social media timeline structures

SAGA: array storage as a DB with support for structural aggregations

Content Provider	ACM Digital Library
Author	Nandi, Arnab Wang, Yi Agrawal, Gagan
Abstract	In recent years, many Array DBMSs, including SciDB and RasDaMan have emerged to meet the needs of data management applications where the natural structures are the arrays. These systems, like their relational counterparts, involve an expensive data ingestion phase. The paradigm of using native storage as a DB and providing database-like support (e.g., the NoDB approach) has recently been shown to be an effective approach for dealing with infrequently queried data, where data ingestion costs cannot be justified, though only in context of relational data. Applications that generate massive arrays, such as the scientific simulations, often store the data in one of a small number of array storage formats, like NetCDF or HDF5. Thus, a natural question is, "can database-like functionality be supported over native array storage?". In this paper, we present algorithms, different partitioning strategies, and an analytical model for supporting structural (grid, sliding, hierarchical, and circular) aggregations over native array storage, and describe implementation of this approach in a system we refer to as Structural AGgregations over Array storage (SAGA). We show how the relative performance of different partitioning strategies changes with varying amount of computation in the aggregation function and different levels of data skew, and our model is effective in choosing the best partitioning strategy. Performance comparison with SciDB shows that despite working on native array storage, the aggregation costs with our system are lower. Finally, we also show that our structural aggregation implementations achieve high parallel efficiency.
Starting Page	1
Ending Page	12
Page Count	12
File Format	PDF
ISBN	9781450327220
DOI	10.1145/2618243.2618270
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2014-06-30
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Scientific databases Array databases Structural aggregations
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

ArrayStore: a storage manager for complex parallel array processing

CROSS-DB: a feature-extended multidimensional data model for statistical and scientific databases

Extending the SQL array concept to support scientific analytics

A storage structure for graph-oriented databases using an array of element types

A Storage Server for the Efficient Support of Complex Objects (1996)

A fixed storage distributed graph database hybrid with at-scale OLAP expression and I/O support of a relational DB: Kinetica-Graph

Hybrid merge/overlap execution technique for parallel array processing

A Storage Server for the Efficient Support of Complex Objects

Efficient Versioning for Scientific Array Databases 1

SAGA: array storage as a DB with support for structural aggregations

Similar Documents

ArrayStore: a storage manager for complex parallel array processing

CROSS-DB: a feature-extended multidimensional data model for statistical and scientific databases

Extending the SQL array concept to support scientific analytics

A storage structure for graph-oriented databases using an array of element types

A Storage Server for the Efficient Support of Complex Objects (1996)

A fixed storage distributed graph database hybrid with at-scale OLAP expression and I/O support of a relational DB: Kinetica-Graph

Hybrid merge/overlap execution technique for parallel array processing

A Storage Server for the Efficient Support of Complex Objects

Efficient Versioning for Scientific Array Databases 1

SAGA: array storage as a DB with support for structural aggregations