NDLI: Communication-Aware Load Balancing for Parallel Applications on Clusters

Content Provider	IEEE Xplore Digital Library
Author	Xiao Qin Hong Jiang Manzanares, A. Xiaojun Ruan Shu Yin
Copyright Year	1968
Abstract	Cluster computing has emerged as a primary and cost-effective platform for running parallel applications, including communication-intensive applications that transfer a large amount of data among the nodes of a cluster via the interconnection network. Conventional load balancers have proven effective in increasing the utilization of CPU, memory, and disk I/O resources in a cluster. However, most of the existing load-balancing schemes ignore network resources, leaving an opportunity to improve the effective bandwidth of networks on clusters running parallel applications. For this reason, we propose a communication-aware load-balancing technique that is capable of improving the performance of communication-intensive applications by increasing the effective utilization of networks in cluster environments. To facilitate the proposed load-balancing scheme, we introduce a behavior model for parallel applications with large requirements of network, CPU, memory, and disk I/O resources. Our load-balancing scheme can make full use of this model to quickly and accurately determine the load induced by a variety of parallel applications. Simulation results generated from a diverse set of both synthetic bulk synchronous and real parallel applications on a cluster show that our scheme significantly improves the performance, in terms of slowdown and turn-around time, over existing schemes by up to 206 percent (with an average of 74 percent) and 235 percent (with an average of 82 percent), respectively.
Sponsorship	IEEE Computer Society Technical Committee on Distributed Process IEEE Computer Society Technical Committee on VLSI IEEE Technical Committee on Computer Architecture IEEE Computer Society
Starting Page	42
Ending Page	52
Page Count	11
File Size	2514887
File Format	PDF
ISSN	00189340
Volume Number	59
Issue Number	1
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-01-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Load modeling Load management Central Processing Unit Data mining Software Bandwidth Computational modeling load balancing. Cluster communication-aware computing parallel computing
Content Type	Text
Resource Type	Article
Subject	Theoretical Computer Science Computational Theory and Mathematics Software Hardware and Architecture

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A super-programming approach for mining association rules in parallel on PC clusters

Resource-aware scientific computation on a heterogeneous cluster

Distributed load balancing for parallel main memory hash join

Communication-Aware Supernode Shape

On the Design of Mutually Aware OptimalPricing and Load Balancing Strategiesfor Grid Computing Systems

EagerMap: A Task Mapping Algorithm to Improve Communication and Load Balancing in Clusters of Multicore Systems

Efficient, proximity-aware load balancing for DHT-based P2P systems

Concurrency extraction via hardware methods executing the static instruction stream

Dynamic load balancing in very large shared-nothing hypercube database computers

Communication-Aware Load Balancing for Parallel Applications on Clusters

Similar Documents

A super-programming approach for mining association rules in parallel on PC clusters

Resource-aware scientific computation on a heterogeneous cluster

Distributed load balancing for parallel main memory hash join

Communication-Aware Supernode Shape

On the Design of Mutually Aware OptimalPricing and Load Balancing Strategiesfor Grid Computing Systems

EagerMap: A Task Mapping Algorithm to Improve Communication and Load Balancing in Clusters of Multicore Systems

Efficient, proximity-aware load balancing for DHT-based P2P systems

Concurrency extraction via hardware methods executing the static instruction stream

Dynamic load balancing in very large shared-nothing hypercube database computers

Communication-Aware Load Balancing for Parallel Applications on Clusters