NDLI: Distributed parallel file system for I/O intensive parallel computing on clusters

Content Provider	IEEE Xplore Digital Library
Author	Dominguez-Dominguez, S. Buenabad-Chavez, J.
Copyright Year	2004
Description	Author affiliation: Departamento de Ingenieria Electrica, Centro de Investigacion y de Estudios Avanzados del IPN, Apartado Postal 14-740, Mexico 07360, D.F. (Dominguez-Dominguez, S.; Buenabad-Chavez, J.)
Abstract	Local area networks are now widely used to run parallel applications. They are particularly suitable to run I/O intensive applications, because each node usually includes disk space. However, programming these applications is rather difficult. The programmer must partition data into disk nudes and, at run time, transfer data from disk space into the memory of each node that uses the data and vice versa. Also, such partitioning gives data a fixed location in disk space, and is usually not adequate for performance because processors do not access mostly data in their local memory or disk, but on disk space in remote nodes. This paper presents a distributed parallel file system that both eases the programming and improves the performance of parallel I/O intensive applications. Our file system eases programming through mapping into memory files of size up to hundreds of Giga bytes. It improves performance through automatically diffusing, or migrating and replicating, data in files to the local memory or local disk of the processors that use the data. Data diffusion occurs under a multiple-readers-single-writer protocol. On applications tested the performance gain can be up to 20 % compared to versions using the MPI file system.
Starting Page	194
Ending Page	199
File Size	1083906
Page Count	6
File Format	PDF
ISBN	0780385314
DOI	10.1109/ICEEE.2004.1433875
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2004-09-08
Publisher Place	Mexico
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	System testing Protocols clusters data diffusion Performance gain Programming profession parallel programming File systems Parallel programming Message passing Parallel processing Parallel file systems Dynamic programming Local area networks
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

The design of VIP-FS:a virtual, parallel file system for high performance parallel and distributed computing

A performance comparison of DSM, PVM, and MPI

VODCA: View-Oriented, Distributed, Cluster-Based Approach to Parallel Computing

Development of an MPI for Power System Distributed Parallel Computing in Wide Area Network with P2P Technology

A distributed Prolog system with AND parallelism

Potential benefits of file migration in a heterogeneous distributed file system

File Creation Strategies in a Distributed Metadata File System

Parallelization Analysis on Clusters of Multicore Nodes Using Shared and Distributed Memory Parallel Computing Models

An adaptive I/O load distribution scheme for distributed systems

Distributed parallel file system for I/O intensive parallel computing on clusters

Similar Documents

The design of VIP-FS:a virtual, parallel file system for high performance parallel and distributed computing

A performance comparison of DSM, PVM, and MPI

VODCA: View-Oriented, Distributed, Cluster-Based Approach to Parallel Computing

Development of an MPI for Power System Distributed Parallel Computing in Wide Area Network with P2P Technology

A distributed Prolog system with AND parallelism

Potential benefits of file migration in a heterogeneous distributed file system

File Creation Strategies in a Distributed Metadata File System

Parallelization Analysis on Clusters of Multicore Nodes Using Shared and Distributed Memory Parallel Computing Models

An adaptive I/O load distribution scheme for distributed systems

Distributed parallel file system for I/O intensive parallel computing on clusters