NDLI: Optimizing Scientific Workflows in the Cloud: A Montage Example

Content Provider	IEEE Xplore Digital Library
Author	Jiang, Qingye Lee, Young Choon Arenaz, Manuel Leslie, Luke M. Zomaya, Albert Y.
Copyright Year	2014
Abstract	As scientific workflows are increasingly deployed in clouds, a myriad of studies have been conducted-including the development of workflow execution systems and scheduling/resource-management algorithms-for optimizing the execution of these workflows. However, the efficacy of most, if not all, of these previous works is limited by the original design and structure of workflow, i.e., Sequential code and few bottleneck tasks. In this paper, we address the optimization of scientific workflow execution in clouds by exploiting multi-core systems with the parallelization of bottleneck tasks. To this end, we develop a workflow visualization toolkit to synthetize resource consumption and data transfer patterns, as well as to identify the bottleneck, of the workflow being studied. Parallelization techniques are then applied to the module that is identified as the bottleneck in order to take full advantage of the underlying multicore computing environment. Testing results with a 6.0-degree Montage example on Amazon EC2 with various configurations show that our optimization of workflows (bottleneck tasks in particular) reduces completion time (or make span) by 21% to 43% depending on the instance type being used to run the workflow, without any impact on the cost.
Starting Page	517
Ending Page	522
File Size	1160051
Page Count	6
File Format	PDF
ISBN	9781479978816
DOI	10.1109/UCC.2014.77
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-12-08
Publisher Place	United Kingdom
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Visualization Processor scheduling Data visualization Educational institutions Data transfer Workflow Vectors Electronic mail Optimization Parallelization
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in