NDLI: A hierarchical approach to reducing communication in parallel graph algorithms

Content Provider	ACM Digital Library
Author	Amato, Nancy M. Rauchwerger, Lawrence
Abstract	Large-scale graph computing has become critical due to the ever-increasing size of data. However, distributed graph computations are limited in their scalability and performance due to the heavy communication inherent in such computations. This is exacerbated in scale-free networks, such as social and web graphs, which contain hub vertices that have large degrees and therefore send a large number of messages over the network. Furthermore, many graph algorithms and computations send the same data to each of the neighbors of a vertex. Our proposed approach recognizes this, and reduces communication performed by the algorithm without change to user-code, through a hierarchical machine model imposed upon the input graph. The hierarchical model takes advantage of locale information of the neighboring vertices to reduce communication, both in message volume and total number of bytes sent. It is also able to better exploit the machine hierarchy to further reduce the communication costs, by aggregating traffic between different levels of the machine hierarchy. Results of an implementation in the STAPL GL shows improved scalability and performance over the traditional level-synchronous approach, with 2.5×-8× improvement for a variety of graph algorithms at 12,000+ cores.
Starting Page	285
Ending Page	286
Page Count	2
File Format	PDF
ISSN	03621340 15581160
DOI	10.1145/2858788.2700994
Journal	ACM SIGPLAN Notices (SIGP)
Volume Number	50
Issue Number	8
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	1983-05-01
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Parallel graph processing Big data Distributed computing Graph analytics
Content Type	Text
Resource Type	Article
Subject	Computer Graphics and Computer-Aided Design Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A hierarchical approach to reducing communication in parallel graph algorithms

Lifting sequential graph algorithms for distributed-memory parallel computation

DIVE: A Graph-Based Visual-Analytics Framework for Big Data

Line-drawing algorithms for parallel machines

A "Design First" Approach to Visualization Innovation

Time Dependent Processing in a Parallel Pipeline Architecture

Distributing data and control for ray tracing in parallel

Parallel volume rendering using binary-swap compositing

Processing Big Data Graphs on Memory-Restricted Systems

A hierarchical approach to reducing communication in parallel graph algorithms

Similar Documents

A hierarchical approach to reducing communication in parallel graph algorithms

Lifting sequential graph algorithms for distributed-memory parallel computation

DIVE: A Graph-Based Visual-Analytics Framework for Big Data

Line-drawing algorithms for parallel machines

A "Design First" Approach to Visualization Innovation

Time Dependent Processing in a Parallel Pipeline Architecture

Distributing data and control for ray tracing in parallel

Parallel volume rendering using binary-swap compositing

Processing Big Data Graphs on Memory-Restricted Systems

A hierarchical approach to reducing communication in parallel graph algorithms