NDLI: On the role of application and resource characterizations in heterogeneous distributed computing systems

Content Provider	Springer Nature Link
Author	Hwang, Eunji Kim, Seontae Kim, Jik Soo Hwang, Soonwook Choi, Young ri
Copyright Year	2016
Abstract	Loosely coupled applications composed of a potentially very large number (from tens of thousands to even billions) of tasks are commonly used in high-throughput computing and many-task computing paradigms. To efficiently execute large-scale computations which can exceed the capability in a single type of computing resources within expected time, we should be able to effectively integrate resources from heterogeneous distributed computing (HDC) systems such as clusters, grids, and clouds. In this paper, we quantitatively analyze the performance of three different real scientific applications consisting of many tasks on top of HDC systems based on a partnership of distributed computing clusters, grids, and clouds to understand the application and resource characteristics, and show practical issues that normal scientific users can face during the course of leveraging these systems. Our experimental study shows that the performance of a loosely coupled application can be significantly affected by the characteristics of a HDC system, along with hardware specification of a node, and their impacts on the performance can vary widely depending on the resource usage pattern of each application. We then devise a preference-based scheduling algorithm that can reflect characteristics and resource usage patterns of various loosely coupled applications running on top of HDC systems from our experimental study. Our preference-based scheduling algorithm can allocate the resources from different HDC systems to loosely coupled applications based on the preferences of the applications for the HDC systems. We evaluate the overall system performance over various preference types, using trace-based simulations, which can be determined based on different factors such as CPU specifications and application throughputs. Our simulation results demonstrate the importance of understanding the application and resource characteristics on effective scheduling of loosely coupled applications on the HDC systems.
Starting Page	2225
Ending Page	2240
Page Count	16
File Format	PDF
ISSN	13867857
Journal	Cluster Computing
Volume Number	19
Issue Number	4
e-ISSN	15737543
Language	English
Publisher	Springer US
Publisher Date	2016-09-15
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Loosely coupled applications High-throughput computing Many-task computing Performance evaluation Heterogeneous distributed computing systems Scheduling algorithm Processor Architectures Operating Systems Computer Communication Networks
Content Type	Text
Resource Type	Article
Subject	Computer Networks and Communications Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Middleware support for many-task computing

MOMTH: multi-objective scheduling algorithm of many tasks in Hadoop

A flexible multi-dimensional QoS performance measure framework for distributed heterogeneous systems

Energy-aware task scheduling in heterogeneous computing environments

HSGA: a hybrid heuristic algorithm for workflow scheduling in cloud systems

Adaptive application-aware job scheduling optimization strategy in heterogeneous infrastructures

A taxonomy of application scheduling tools for high performance cluster computing

A task scheduling algorithm for arbitrarily-connected processors with awareness of link contention

Empirical performance evaluation of schedulers for cluster of workstations

On the role of application and resource characterizations in heterogeneous distributed computing systems

Similar Documents

Middleware support for many-task computing

MOMTH: multi-objective scheduling algorithm of many tasks in Hadoop

A flexible multi-dimensional QoS performance measure framework for distributed heterogeneous systems

Energy-aware task scheduling in heterogeneous computing environments

HSGA: a hybrid heuristic algorithm for workflow scheduling in cloud systems

Adaptive application-aware job scheduling optimization strategy in heterogeneous infrastructures

A taxonomy of application scheduling tools for high performance cluster computing

A task scheduling algorithm for arbitrarily-connected processors with awareness of link contention

Empirical performance evaluation of schedulers for cluster of workstations

On the role of application and resource characterizations in heterogeneous distributed computing systems