NDLI: Exploiting Process Imbalance to Improve MPI Collective Operations in Hierarchical Systems

Please wait, while we are loading the content...

Streaming Task Parallelism

Automatic Parallelization of Kernels in Shared-Memory Multi-GPU Nodes

PaCMap: Topology Mapping of Unstructured Communication Patterns onto Non-contiguous Allocations

Locality-Driven Dynamic GPU Cache Bypassing

Automatic Selection of Sparse Matrix Representation on GPUs

Datacenter Efficiency: What's Next?

Towards Lightweight and Swift Storage Resource Management in Big Data Cloud Era

MODESTO: Data-centric Analytic Optimization of Complex Stencil Programs on Heterogeneous Architectures

GreenPar: Scheduling Parallel High Performance Applications in Green Datacenters

History-Assisted Adaptive-Granularity Caches (HAAG\$) for High Performance 3D DRAM Architectures

Automatically Scalable Computation

Mower: A New Design for Non-blocking Misprediction Recovery

PALMOS: A Transparent, Multi-tasking Acceleration Layer for Parallel Heterogeneous Systems

CSR5: An Efficient Storage Format for Cross-Platform Sparse Matrix-Vector Multiplication

Automatic Energy Efficient Parallelization of Uniform Dependence Computations

COMPASS: A Framework for Automated Performance Modeling and Prediction

A Stall-Aware Warp Scheduling for Dynamically Optimizing Thread-level Parallelism in GPGPUs

Unique Worker model for OpenMP

SemCache++: Semantics-Aware Caching for Efficient Multi-GPU Offloading

Fine-Grained Synchronizations and Dataflow Programming on GPUs

Hadoop+: Modeling and Evaluating the Heterogeneity for MapReduce Applications in Heterogeneous Clusters

FAST: A Fast Stencil Autotuning Framework Based On An Optimal-solution Space Model

Underprovisioning the Grid Power Infrastructure for Green Datacenters

Real-Time In-Memory Checkpointing for Future Hybrid Memory Systems

zFENCE: Data-less Coherence for Efficient Fences

A Nested Partitioning Algorithm for Adaptive Meshes on Heterogeneous Clusters

Optimistic Delinearization of Parametrically Sized Arrays

ASPaS: A Framework for Automatic SIMDization of Parallel Sorting on x86-based Many-core Processors

Composing Algorithmic Skeletons to Express High-Performance Scientific Applications

PeerWave: Exploiting Wavefront Parallelism on GPUs with Peer-SM Synchronization

Exploiting Process Imbalance to Improve MPI Collective Operations in Hierarchical Systems

DaCache: Memory Divergence-Aware GPU Cache Management

Enabling and Exploiting Flexible Task Assignment on GPU through SM-Centric Program Transformations

Active Access: A Mechanism for High-Performance Distributed Data-Centric Computations

Parameterized Diamond Tiling for Stencil Computations with Chapel parallel iterators

Building Fuel Powered Supercomputing Data Center at Low Cost

Leveraging Silicon-Photonic NoC for Designing Scalable GPUs

Criticality-Aware Dynamic Task Scheduling for Heterogeneous Architectures

DASX: Hardware Accelerator for Software Data Structures

Optimizing Overlapped Memory Accesses in User-directed Vectorization

STAPL-RTS: An Application Driven Runtime System

Exascaling Your Library: Will Your Implementation Meet Your Expectations?

Quantifying Performance Bottlenecks of Stencil Computations Using the Execution-Cache-Memory Model

Exploiting Process Imbalance to Improve MPI Collective Operations in Hierarchical Systems

Content Provider	ACM Digital Library
Author	Parsons, Benjamin S. Pai, Vijay S.
Abstract	This work improves the performance of MPI collective communication operations in the presence of imbalanced process arrival times. High performance collective communications are crucial for the performance and scalability of applications, and imbalanced process arrival times are common in these applications. A micro-benchmark is used to investigate the nature of process imbalance with perfectly balanced workloads, and understand the nature of inter- versus intra-node imbalance. These insights are then used to develop imbalance-tolerant reduction, broadcast, and all-to-all algorithms, which minimize the synchronization delay observed by early arriving processes. These algorithms have been implemented and tested on a Cray XE6 using up to 32k cores with varying buffer sizes and levels of imbalance. Results show speedups over MPICH averaging 18.9x for reduce, 5.3x for broadcast, and 6.9x for all-to-all in the presence of high, but not unreasonable, imbalance.
Starting Page	57
Ending Page	66
Page Count	10
File Format	PDF
ISBN	9781450335591
DOI	10.1145/2751205.2751242
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2015-06-08
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Process imbalance Collective communication Mpi Hpc
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Tolerating Process Imbalance in Hierarchical Collective Operations

A study of process arrival patterns for MPI collective operations ( ICS '07 )

Kernel-Assisted MPI Collective Communication among Many-core Clusters

Process Distance-Aware Adaptive MPI Collective Communications

Implementation and performance analysis of non-blocking collective operations for MPI

Accelerating MPI collective communications through hierarchical algorithms with flexible inter-node communication and imbalance awareness

Transforming blocking MPI collectives to Non-blocking and persistent operations

Adaptive Selection of Communication Methods to Optimize Collective MPI Operations (2006)

Automatic tuning of collective communication operations in mpi (2003).

Exploiting Process Imbalance to Improve MPI Collective Operations in Hierarchical Systems

Similar Documents

Tolerating Process Imbalance in Hierarchical Collective Operations

A study of process arrival patterns for MPI collective operations ( ICS '07 )

Kernel-Assisted MPI Collective Communication among Many-core Clusters

Process Distance-Aware Adaptive MPI Collective Communications

Implementation and performance analysis of non-blocking collective operations for MPI

Accelerating MPI collective communications through hierarchical algorithms with flexible inter-node communication and imbalance awareness

Transforming blocking MPI collectives to Non-blocking and persistent operations

Adaptive Selection of Communication Methods to Optimize Collective MPI Operations (2006)

Automatic tuning of collective communication operations in mpi (2003).

Exploiting Process Imbalance to Improve MPI Collective Operations in Hierarchical Systems