NDLI: Parallel I/O performance of fine grained data distributions

Content Provider	IEEE Xplore Digital Library
Author	Yong Cho Winslett, M. Ying Chen Szu-Wen Kuo
Copyright Year	1998
Description	Author affiliation: Dept. of Comput. Sci., Illinois Univ., Urbana, IL, USA (Yong Cho)
Abstract	Fine grained data distributions are widely used to balance computational loads across compute processes in parallel scientific applications. When a fine grained data distribution is used in memory, performance of I/O intensive applications can be limited not only by disk speed but also by message passing, because a large number of small messages may be generated by the implementation strategy used in the underlying parallel file system or parallel I/O library. Combining (or packetizing) a set of small messages into a large message is generally known to speed up parallel I/O. However, overall I/O performance is affected not only by small messages but also by other factors like cyclic block size and interconnect characteristics. We describe small message combination and communication scheduling for fine grained data distributions in the Panda parallel I/O library and analyze I/O performance on parallel platforms having different interconnects: IBM SP2, IBM workstation cluster connected by FDDI and Pentium II cluster connected by Myrinet.
Starting Page	163
Ending Page	170
File Size	146610
Page Count	8
File Format	PDF
ISBN	0818685794
ISSN	10828907
DOI	10.1109/HPDC.1998.709969
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	1998-07-31
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Concurrent computing High performance computing File systems Libraries Computer science Application software Message passing Identity-based encryption FDDI Degradation
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Collective caching: application-aware client-side file caching

Using Object Based Files for High Performance Parallel I/O

Xen-Based HPC: A Parallel I/O Perspective

Implementation and Performance Evaluation of Socket and RMI based Java Message Passing Systems

SunTM MPI I/O: Efficient I/O for Parallel Applications

Scaling parallel I/O performance through I/O delegate and caching system

TFLOPS PFS: Architecture and Design of a Highly Efficient Parallel File System

Design and evaluation of a high performance file system for message passing parallel computers

One-Sided Message Passing Programming Model for Unstructured Peer-to-Peer Computing System

Parallel I/O performance of fine grained data distributions

Similar Documents

Collective caching: application-aware client-side file caching

Using Object Based Files for High Performance Parallel I/O

Xen-Based HPC: A Parallel I/O Perspective

Implementation and Performance Evaluation of Socket and RMI based Java Message Passing Systems

SunTM MPI I/O: Efficient I/O for Parallel Applications

Scaling parallel I/O performance through I/O delegate and caching system

TFLOPS PFS: Architecture and Design of a Highly Efficient Parallel File System

Design and evaluation of a high performance file system for message passing parallel computers

One-Sided Message Passing Programming Model for Unstructured Peer-to-Peer Computing System

Parallel I/O performance of fine grained data distributions