NDLI: Sparse glider datasets: A case study for NoSQL databases

Content Provider	IEEE Xplore Digital Library
Author	Lindemuth, M. Lembke, C.
Copyright Year	2013
Description	Author affiliation: Coll. of Marine Sci. Ocean Technol. Group, Univ. of South Florida, St. Petersburg, FL, USA (Lindemuth, M.; Lembke, C.)
Abstract	Multi-sensor platforms like buoys and gliders produce one or more readings per sensor on varying, discrete time frequencies. The resulting datasets are a matricies with rows containing readings from sensors that reported at a moment in time and NULL for missing readings from sensors that did not. Traditional Relational Database Management Systems (RDBMS) are already well suited for the dense matricies in which NULL values are infrequent. The efficiency of these systems deteriorates though as data becomes more sparse. The University of South Florida College of Marine Science Ocean Technology Group (COT) operates four gliders. Each glider produces dynamic, different sparse datasets. Other data management solutions exist, but they are based on a RDBMS. COT has been investigating an alternative without using and RDBMS. Glider Database Alternative with Mongo (GDAM) is a data management system for gliders built on the MongoDB NoSQL database engine. It is live in production at COT. GDAM is a collection of scripts which parse, process and store real-time glider datasets. Data is parsed as soon as it is transmitted via satellite to our shore-based servers. The system has been tested during two Slocum G1 glider deployments in September and October of 2012. Archival datasets dating back to March of 2009 have also been uploaded into this system. Records are indexed by time, GPS, and depth with the ability to add more indexes as necessary. The paper outlines dataset problems identified using data from COT glider operations in 2012. These problems inform a discussion of design decisions and possible options considering both RDBMS and NoSQL systems. The paper concludes by discussing the current implementation of GDAM.
Sponsorship	IEEE Oceanic Eng. Soc.
Starting Page	1
Ending Page	6
File Size	784581
Page Count	6
File Format	PDF
ISBN	9780933957404
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-09-23
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	MTS
Subject Keyword	Servers Educational institutions Relational databases Indexes Marine technology
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in