NDLI: Exploiting in-memory storage for improving workflow executions in cloud platforms

Content Provider	Springer Nature Link
Author	Rodrigo Duro, Francisco Marozzo, Fabrizio Garcia Blas, Javier Talia, Domenico Trunfio, Paolo
Copyright Year	2016
Abstract	The Data Mining Cloud Framework (DMCF) is an environment for designing and executing data analysis workflows in cloud platforms. Currently, DMCF relies on the default storage of the public cloud provider for any I/O-related operation. This implies that the I/O performance of DMCF is limited by the performance of the default storage. In this work, we propose the usage of the Hercules system within DMCF as an ad hoc storage system for temporary data produced inside workflow-based applications. Hercules is a distributed in-memory storage system highly scalable and easy to deploy. The proposed solution takes advantage of the scalability capabilities of Hercules to avoid the bandwidth limits of the default storage. We evaluated the performance of Hercules compared with the Microsoft Azure Storage solution by using synthetic benchmarks with the objective of demonstrating the viability of the proposed solution. Then, we evaluated the integration of Hercules and DMCF on a real application consisting of a workflow that accesses temporary data using either Azure storage or Hercules. The I/O overhead in this real-life scenario using Hercules has been reduced by 36 % with respect to Azure storage, leading to a 13 % reduction of the total execution time. This confirms that our in-memory approach is effective in improving the performance of data-intensive workflow executions in cloud-based platforms.
Starting Page	4069
Ending Page	4088
Page Count	20
File Format	PDF
ISSN	09208542
Journal	The Journal of Supercomputing
Volume Number	72
Issue Number	11
e-ISSN	15730484
Language	English
Publisher	Springer US
Publisher Date	2016-02-27
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	DMCF Hercules Workflows In-memory storage Data cache Microsoft Azure Programming Languages, Compilers, Interpreters Processor Architectures Computer Science
Content Type	Text
Resource Type	Article
Subject	Theoretical Computer Science Information Systems Hardware and Architecture Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Dynamic Partitioning of Shared Cache Memory

Exploiting application-level similarity to improve SSD cache performance in Hadoop

A market-oriented hierarchical scheduling strategy in cloud workflow systems

An efficient cloud storage system for tele-health services

Enabling workflows in GridSolve: request sequencing and service trading

Active memory controller

Workflow scheduling in cloud: a survey

Architecture for speeding up program execution with cloud technology

SAPDS: self-healing attribute-based privacy aware data sharing in cloud

Exploiting in-memory storage for improving workflow executions in cloud platforms

Similar Documents

Dynamic Partitioning of Shared Cache Memory

Exploiting application-level similarity to improve SSD cache performance in Hadoop

A market-oriented hierarchical scheduling strategy in cloud workflow systems

An efficient cloud storage system for tele-health services

Enabling workflows in GridSolve: request sequencing and service trading

Active memory controller

Workflow scheduling in cloud: a survey

Architecture for speeding up program execution with cloud technology

SAPDS: self-healing attribute-based privacy aware data sharing in cloud

Exploiting in-memory storage for improving workflow executions in cloud platforms