NDLI: Exploring the All-to-All Collective Optimization Space with ConnectX CORE-Direct

Content Provider	IEEE Xplore Digital Library
Author	Venkata, M.G. Graham, R.L. Ladd, J. Shamis, P.
Copyright Year	2012
Abstract	The all-to-all collective communication operation is used by many scientific applications, and is one of the most time consuming and challenging collective operation to optimize. The algorithms for all-to-all operations typically fall into two classes, logarithmic and linear scaling algorithms, with Bruck's algorithm, a logarithmic scaling algorithm, used in many small-data all-to-all implementations. The recent addition of InfiniBand CORE-Direct support for network management of collective communications offers new opportunities for optimizing all-to-all operation as well as supporting truly asynchronous implementations of these operations. This paper presents several new enhancements to the Bruck small-data algorithm that leverage CORE-Direct and other InfiniBand network capabilities to produce efficient implementations of this collective operation. These include RDMA, SR-RNR, and SR-RTR algorithms. In addition, nonblocking implementations of these collective operations are also presented. Benchmark results show that the RDMA algorithm, which uses CORE-Direct capabilities to offload collective communication management to the Host Channel Adapter (HCA), hardware gather support for sending non-continuous data, and low-latency RDMA semantics, performs the best. For a 64 processes and 128 byte-per-process all-to-all, the RDMA algorithm performs 27% better than Bruck's algorithm implementation in Open MPI and 136% better than the SR-RTR algorithm. In addition, the nonblocking versions of these algorithms have the same performance characteristics as the blocking algorithms. Finally, measurements of computation/communication overlap capacity show that all offloaded algorithms achieve about 98% overlap for large data all-to-all, whereas implementations using host-based progress achieve only about 9.5% overlap.
Starting Page	289
Ending Page	298
File Size	403348
Page Count	10
File Format	PDF
ISBN	9781467325080
ISSN	01903918
DOI	10.1109/ICPP.2012.28
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-09-10
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Receivers Algorithm design and analysis Benchmark testing Optimization Hardware Semantics ConnectX Core-Direct Collective Operations Alltoall Communication MPI High Performance Computing InfiniBand
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

ConnectX-2 CORE-Direct Enabled Asynchronous Broadcast Collective Communications

High Performance Alltoall and Allgather Designs for InfiniBand MIC Clusters

PERFORMANCE ANALYSIS OF MESSAGE PASSING INTERFACE COLLECTIVE COMMUNICATION ON INTEL XEON QUAD-CORE GIGABIT ETHERNET AND INFINIBAND CLUSTERS

Designing Non-blocking Broadcast with Collective Offload on InfiniBand Clusters: A Case Study with HPL

Designing efficient small message transfer mechanism for inter-node MPI communication on InfiniBand GPU clusters

Designing Topology-Aware Communication Schedules for Alltoall Operations in Large InfiniBand Clusters

Exploring Infiniband Hardware Virtualization in OpenNebula towards Efficient High-Performance Computing

Improving Application Performance and Predictability Using Multiple Virtual Lanes in Modern Multi-core InfiniBand Clusters

MPI Alltoall Personalized Exchange on GPGPU Clusters: Design Alternatives and Benefit

Exploring the All-to-All Collective Optimization Space with ConnectX CORE-Direct

Similar Documents

ConnectX-2 CORE-Direct Enabled Asynchronous Broadcast Collective Communications

High Performance Alltoall and Allgather Designs for InfiniBand MIC Clusters

PERFORMANCE ANALYSIS OF MESSAGE PASSING INTERFACE COLLECTIVE COMMUNICATION ON INTEL XEON QUAD-CORE GIGABIT ETHERNET AND INFINIBAND CLUSTERS

Designing Non-blocking Broadcast with Collective Offload on InfiniBand Clusters: A Case Study with HPL

Designing efficient small message transfer mechanism for inter-node MPI communication on InfiniBand GPU clusters

Designing Topology-Aware Communication Schedules for Alltoall Operations in Large InfiniBand Clusters

Exploring Infiniband Hardware Virtualization in OpenNebula towards Efficient High-Performance Computing

Improving Application Performance and Predictability Using Multiple Virtual Lanes in Modern Multi-core InfiniBand Clusters

MPI Alltoall Personalized Exchange on GPGPU Clusters: Design Alternatives and Benefit

Exploring the All-to-All Collective Optimization Space with ConnectX CORE-Direct