NDLI: Comparison of resource platform selection approaches for scientific workflows

Please wait, while we are loading the content...

Horizon: efficient deadline-driven disk I/O management for distributed storage systems

DataSpaces: an interaction and coordination framework for coupled simulation workflows

Software architecture definition for on-demand cloud provisioning

MOON: MapReduce On Opportunistic eNvironments

Thermal aware server provisioning and workload distribution for internet data centers

A GPU accelerated storage system

Dimension reduction and visualization of large high-dimensional data via interpolation

PV-EASY: a strict fairness guaranteed and prediction enabled scheduler in parallel job scheduling

Impact of sub-optimal checkpoint intervals on application efficiency in computational clusters

Early observations on the performance of Windows Azure

Cloud computing paradigms for pleasingly parallel biomedical applications

Optimal enterprise data architecture using publish and subscribe

Netlag: a performance evaluation tool for massively multi-user networked applications

Efficient querying of distributed provenance stores

Cluster-wide context switch of virtualized jobs

Characterising a grid site's traffic

Parallel processing of data from very large-scale wireless sensor networks

Run-time optimizations for replicated dataflows on heterogeneous environments

ParaTrac: a fine-grained profiler for data-intensive workflows

High occupancy resource allocation for grid and cloud systems, a study with DRIVE

MRAP: a novel MapReduce-based framework to support HPC analytics applications with access patterns

I/O scheduling model of virtual machine based on multi-core dynamic partitioning

Computation mapping for multi-level storage cache hierarchies

New caching techniques for web search engines

XCo: explicit coordination to prevent network fabric congestion in cloud computing cluster platforms

OpenGL application live migration with GPU acceleration in personal cloud

An overview of the Open Science Data Cloud

MPIPairwiseStatSig: parallel pairwise statistical significance estimation of local sequence alignment

The development of QC standards for ARM data products

BTWorld: towards observing the global BitTorrent file-sharing network

SAGA-based file access application over multi-filesystem middleware

Pools of virtual boxes: building campus grids with virtual machines

A data transfer framework for large-scale science experiments

Massive Semantic Web data compression with MapReduce

Performance analysis of dynamic workflow scheduling in multicluster grids

Highly available component sharing in large-scale multi-tenant cloud systems

Data centric highly parallel debugging

A practical way to extend shared memory support beyond a motherboard at low cost

Cashing in on hints for better prefetching and caching in PVFS and MPI-IO

Mendel: efficiently verifying the lineage of data modified in multiple trust domains

Scalability of communicators and groups in MPI

A service composition framework for market-oriented high performance computing cloud

Providing a cloud network infrastructure on a supercomputer

Exploring the RNA folding energy landscape using scalable distributed cyberinfrastructure

A quality screening service for remote sensing data

A hybrid Markov chain model for workload on parallel computers

Toward high performance computing in unconventional computing environments

Janus: a cross-layer soft real-time architecture for virtualization

Towards optimising distributed data streaming graphs using parallel streams

Very large pattern databases for heuristic search

GPU-based parallel householder bidiagonalization

Case study for running HPC applications in public clouds

Biocompute: towards a collaborative workspace for data intensive bio-science

Monitoring data quality in Kepler

LogGOPSim: simulating large-scale applications in the LogGOPS model

Weaver: integrating distributed computing abstractions into scientific workflows using Python

DistriBit: a distributed dynamic binary translator system for thin client computing

Detouring and replication for fast and reliable internet-scale stream processing

Twister: a runtime for iterative MapReduce

Authorization within grid-computing using certificateless identity-based proxy signature

Exploring application and infrastructure adaptation on hybrid grid-cloud infrastructure

Modeling sequence and function similarity between proteins for protein functional annotation

Towards long term data quality in a large scale biometrics experiment

Fast and scalable simulation of volunteer computing systems using SimGrid

A distributed workflow for an astrophysical OpenMP application: using the data capacitor over WAN to enhance productivity

Scaling virtual organization clusters over a wide area network using the Kestrel workload management system

File-access patterns of data-intensive workflow applications and their implications to distributed filesystems

Pydoop: a Python MapReduce and HDFS API for Hadoop

A global address space framework for irregular applications

AzureBlast: a case study of developing science applications on the cloud

Browsing large scale cheminformatics data with dimension reduction

Design, implementation and use of a simulation data archive for coastal science

Storage deduplication for Virtual Ad Hoc Network testbed by File-level Block Sharing

Versioning for workflow evolution

Pairwise Element Computation with MapReduce

Improving host swapping using adaptive prefetching and paging notifier

Seeking supernovae in the clouds: a performance study

Data parallelism in bioinformatics workflows using Hydra

ROARS: a scalable repository for data intensive scientific computing

Improving the Hadoop map/reduce framework to support concurrent appends through the BlobSeer BLOB management system

Resource provisioning with budget constraints for adaptive applications in cloud environments

Migrating a (large) science database to the cloud

Optimization of a parallel permutation testing function for the SPRINT R package

GatorShare: a file system framework for high-throughput data management

Multi-GPU volume rendering using MapReduce

SoftPower: fine-grain power estimations using performance counters

Reshaping text data for efficient processing on Amazon EC2

Text mining for bone biology

MR-scope: a real-time tracing tool for MapReduce

Simultaneous performance exploration and optimized search with volunteer computing

Comparison of resource platform selection approaches for scientific workflows

CUDA-based triangulations of convolution molecular surfaces

Parallelizing multiple group-by query in share-nothing environment: a MapReduce study case

Lessons learned from moving earth system grid data sets over a 20 Gbps wide-area network

A perspective on scientific cloud computing

Towards energy-aware autonomic provisioning for virtualized environments

Distributing the power of OLAP

Morco: middleware framework for long-running multi-component applications on batch grids

Lightning: self-adaptive, energy-conserving, multi-zoned, commodity green cloud storage system

Early defense: enabling attribute-based authorization in Grid firewalls

Design and evaluation of a self-healing Kepler for scientific workflows

ParaText: scalable text modeling and analysis

DiscFinder: a data-intensive scalable cluster finder for astrophysics

Improving MPI communication via data type fission

Pwrake: a parallel and distributed flexible workflow management tool for wide-area data intensive computing

A layout-aware optimization strategy for collective I/O

Comparison of resource platform selection approaches for scientific workflows

Content Provider	ACM Digital Library
Author	Simmhan, Yogesh Ramakrishnan, Lavanya
Abstract	Cloud computing is increasingly considered as an additional computational resource platform for scientific workflows. The cloud offers opportunity to scale-out applications from desktops and local cluster resources. Each platform has different properties (e.g., queue wait times in high performance systems, virtual machine startup overhead in clouds) and characteristics (e.g., custom environments in cloud) that makes choosing from these diverse resource platforms for a workflow execution a challenge for scientists. Scientists are often faced with deciding resource platform selection trade-offs with limited information on the actual workflows. While many workflow planning methods have explored resource selection or task scheduling, these methods often require fine-scale characterization of the workflow that is onerous for a scientist. In this paper, we describe our early exploratory work in using blackbox characteristics for a cost-benefit analysis of using different resource platforms. In our blackbox method, we use only limited high-level information on the workflow length, width, and data sizes. The length and width are indicative of the workflow duration and parallelism. We compare the effectiveness of this approach to other resource selection models using two exemplar scientific workflows on desktop, local cluster, HPC center, and cloud platforms. Early results suggest that the blackbox model often makes the same resource selections as a more fine-grained whitebox model. We believe the simplicity of the blackbox model can help inform a scientist on the applicability of a new resource platform, such as cloud resources, even before porting an existing workflow.
Starting Page	445
Ending Page	450
Page Count	6
File Format	PDF
ISBN	9781605589428
DOI	10.1145/1851476.1851541
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2010-06-21
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Cloud computing High performance computing Workflow structure Escience Resource management Workflow patterns
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in