NDLI: PStore: an efficient storage framework for managing scientific data

Please wait, while we are loading the content...

Maintaining a microbial genome & metagenome data analysis system in an academic setting

Offline cleaning of RFID trajectory data

SAGA: array storage as a DB with support for structural aggregations

Data perturbation for outlier detection ensembles

(k, d)-core anonymity: structural anonymization of massive networks

SensorBench: benchmarking approaches to processing wireless sensor network data

PStore: an efficient storage framework for managing scientific data

Geometric graph matching and similarity: a probabilistic approach

A case study in optimizing continuous queries using the magic update technique

DistillFlow: removing redundancy in scientific workflows

Efficient classification of billions of points into complex geographic regions using hierarchical triangular mesh

Skew-resistant parallel in-memory spatial join

Extending the SQL array concept to support scientific analytics

A subspace filter supporting the discovery of small clusters in very noisy datasets

Matching dominance: capture the semantics of dominance for multi-dimensional uncertain objects

Managing evolving shapes in sensor networks

Inverse predictions on continuous models in scientific databases

Schema matching over relations, attributes, and data values

Node classification in uncertain graphs

MR-microT: a MapReduce-based MicroRNA target prediction method

Mining statistically sound co-location patterns at multiple distances

SLACID - sparse linear algebra in a column-oriented in-memory database system

DivIDE: efficient diversification for interactive data exploration

A provable algorithmic approach to product selection problems for market entry and sustainability

Proactive adaptations in sensor network query processing

Helping scientists reconnect their datasets

Point cloud databases

Integrating fault-tolerance and elasticity in a distributed data stream processing system

Integrating non-spatial preferences into spatial location queries

Efficient data management and statistics with zero-copy integration

Local context selection for outlier ranking in graphs with multiple numeric node attributes

Distributed data placement to minimize communication costs via graph partitioning

Boosting the basic counting on distributed streams

Detecting correlated columns in relational databases with mixed data types

Toward efficient and reliable genome analysis using main-memory database systems

Simulation workflow design tailor-made for scientists

Efficient processing of exploratory top-k joins

A system for efficient and simultaneous processing of moving K nearest neighbor and spatial keyword queries

A study of partitioning and parallel UDF execution with the SAP HANA database

Subspace anytime stream clustering

Efficient temporal shortest path queries on evolving social graphs

Data movement in hybrid analytic systems: a case for automation

Protection of sensitive trajectory datasets through spatial and temporal exchange

New approaches to storing and manipulating multi-dimensional sparse arrays

Exploring subspace clustering for recommendations

Data patterns to alleviate the design of scientific workflows exemplified by a bone simulation

Communication-efficient preference top-k monitoring queries via subscriptions

On efficiently generating realistic social media timeline structures

PStore: an efficient storage framework for managing scientific data

Content Provider	ACM Digital Library
Author	Bhattacherjee, Souvik Sussman, Alan Deshpande, Amol
Abstract	In this paper, we present the design, implementation, and evaluation of PStore, a no-overwrite storage framework for managing large volumes of array data generated by scientific simulations. PStore consists of two modules, a data ingestion module and a query processing module, that respectively address two of the key challenges in scientific simulation data management. The data ingestion module is geared toward handling the high volumes of simulation data generated at a very rapid rate, which often makes it impossible to offload the data onto storage devices; the module is responsible for selecting an appropriate compression scheme for the data at hand, chunking the data, and then compressing it before sending it to the storage nodes. On the other hand, the query processing module is in charge of efficiently executing different types of queries over the stored data; in this paper, we specifically focus on dicing (also called range) queries. PStore provides a suite of compression schemes that leverage, and in some cases extend, existing techniques to provide support for diverse scientific simulation data. To efficiently execute queries over such compressed data, PStore adopts and extends a two-level chunking scheme by incorporating the effect of compression, and hides expensive disk latencies for long running range queries by exploiting chunk prefetching. In addition, we also parallelize the query processing module to further speed up execution. We evaluate PStore on a 140 GB dataset obtained from real-world simulations using the regional climate model CWRF [5]. In this paper, we use both 3D and 4D datasets and demonstrate high performance through extensive experiments.
Starting Page	1
Ending Page	12
Page Count	12
File Format	PDF
ISBN	9781450327220
DOI	10.1145/2618243.2618268
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2014-06-30
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Range query Two-level chunking Compression Byte-wise partitioning Array data Floating point data
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in