NDLI: A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds

Content Provider	Springer Nature Link
Author	Oliveira, Daniel Ocaña, Kary A. C. S. Baião, Fernanda Mattoso, Marta
Copyright Year	2012
Abstract	In the last years, scientific workflows have emerged as a fundamental abstraction for structuring and executing scientific experiments in computational environments. Scientific workflows are becoming increasingly complex and more demanding in terms of computational resources, thus requiring the usage of parallel techniques and high performance computing (HPC) environments. Meanwhile, clouds have emerged as a new paradigm where resources are virtualized and provided on demand. By using clouds, scientists have expanded beyond single parallel computers to hundreds or even thousands of virtual machines. Although the initial focus of clouds was to provide high throughput computing, clouds are already being used to provide an HPC environment where elastic resources can be instantiated on demand during the course of a scientific workflow. However, this model also raises many open, yet important, challenges such as scheduling workflow activities. Scheduling parallel scientific workflows in the cloud is a very complex task since we have to take into account many different criteria and to explore the elasticity characteristic for optimizing workflow execution. In this paper, we introduce an adaptive scheduling heuristic for parallel execution of scientific workflows in the cloud that is based on three criteria: total execution time (makespan), reliability and financial cost. Besides scheduling workflow activities based on a 3-objective cost model, this approach also scales resources up and down according to the restrictions imposed by scientists before workflow execution. This tuning is based on provenance data captured and queried at runtime. We conducted a thorough validation of our approach using a real bioinformatics workflow. The experiments were performed in SciCumulus, a cloud workflow engine for managing scientific workflow execution.
Starting Page	521
Ending Page	552
Page Count	32
File Format	PDF
ISSN	15707873
Journal	Journal of Grid Computing
Volume Number	10
Issue Number	3
e-ISSN	15729184
Language	English
Publisher	Springer Netherlands
Publisher Date	2012-08-25
Publisher Place	Dordrecht
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Cloud computing Scientific workflow Scientific experiment Provenance Management of Computing and Information Systems Processor Architectures User Interfaces and Human Computer Interaction
Content Type	Text
Resource Type	Article
Subject	Computer Networks and Communications Information Systems Software Hardware and Architecture

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

A Survey of Data-Intensive Scientific Workflow Management

Cost-Time Efficient Scheduling Plan for Executing Workflows in the Cloud

Towards the Scheduling of Multiple Workflows on Computational Grids

Science in the Cloud: Allocation and Execution of Data-Intensive Scientific Workflows

A Dynamic Cloud Dimensioning Approach for Parallel Scientific Workflows: a Case Study in the Comparative Genomics Domain

Infrastructure Aware Scientific Workflows and Infrastructure Aware Workflow Managers in Science Gateways

An Evaluation of the Cost and Performance of Scientific Workflows on Amazon EC2

Bundle and Pool Architecture for Multi-Language, Robust, Scalable Workflow Executions

A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds

Similar Documents

Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

A Survey of Data-Intensive Scientific Workflow Management

Cost-Time Efficient Scheduling Plan for Executing Workflows in the Cloud

Towards the Scheduling of Multiple Workflows on Computational Grids

Science in the Cloud: Allocation and Execution of Data-Intensive Scientific Workflows

A Dynamic Cloud Dimensioning Approach for Parallel Scientific Workflows: a Case Study in the Comparative Genomics Domain

Infrastructure Aware Scientific Workflows and Infrastructure Aware Workflow Managers in Science Gateways

An Evaluation of the Cost and Performance of Scientific Workflows on Amazon EC2

Bundle and Pool Architecture for Multi-Language, Robust, Scalable Workflow Executions

A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds