NDLI: Many task computing for multidisciplinary ocean sciences: real-time uncertainty prediction and data assimilation

Please wait, while we are loading the content...

Robust workflows for science and engineering

Exploring many task computing in scientific workflows

Lessons learned from a year's worth of benchmarks of large data clouds

A data throughput prediction and optimization service for widely distributed many-task computing

Evaluating use of data flow systems for large graph analysis

Cloud technologies for bioinformatics applications

In cloud, do MTC or HTC service providers benefit from the economies of scale?

Nephele: efficient parallel data processing in the cloud

Scalable computing with parallel tasks

OddCI: on-demand distributed computing infrastructure

Kestrel: an XMPP-based framework for many task computing applications

Highly scalable genome assembly on campus grids

Ensemble dispatching on an IBM Blue Gene/L for a bioinformatics knowledge environment

Many task computing for multidisciplinary ocean sciences: real-time uncertainty prediction and data assimilation

Many task computing for multidisciplinary ocean sciences: real-time uncertainty prediction and data assimilation

Content Provider	ACM Digital Library
Author	Lermusiaux, Pierre F. J. Evangelinos, Constantinos Hill, Chris N. Xu, Jinshan Haley, Patrick J.
Abstract	Error Subspace Statistical Estimation (ESSE), an uncertainty prediction and data assimilation methodology employed for real-time ocean forecasts, is based on a characterization and prediction of the largest uncertainties. This is carried out by evolving an error subspace of variable size. We use an ensemble of stochastic model simulations, initialized based on an estimate of the dominant initial uncertainties, to predict the error subspace of the model fields. The dominant error covariance (generated via an SVD of the ensemble-generated error covariance matrix) is used for data assimilation. The resulting ocean fields are provided as the input to acoustic modeling, allowing for the prediction and study of the spatiotemporal variations in acoustic propagation and their uncertainties. The ESSE procedure is a classic case of Many Task Computing: These codes are managed based on dynamic workflows for the: (i) perturbation of the initial mean state, (ii) subsequent ensemble of stochastic PE model runs, (iii) continuous generation of the covariance matrix, (iv) successive computations of the SVD of the ensemble spread until a convergence criterion is satisfied, and (v) data assimilation. Its ensemble nature makes it a many task data intensive application and its dynamic workflow gives it heterogeneity. Subsequent acoustics propagation modeling involves a very large ensemble of short-in-duration acoustics runs. We study the execution characteristics and challenges of a distributed ESSE workflow on a large dedicated cluster and the usability of enhancing this with runs on Amazon EC2 and the Teragrid and the I/O challenges faced.
Starting Page	1
Ending Page	10
Page Count	10
File Format	PDF
ISBN	9781605587141
DOI	10.1145/1646468.1646482
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2009-11-16
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Assimilation Ensemble Data-intensive Mtc
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in