NDLI: On-demand file staging system for Linux clusters

Content Provider	IEEE Xplore Digital Library
Author	Hori, A. Kamoshida, Y. Matsuba, H. Ohta, K. Yasui, T. Sumimoto, S. Ishikawa, Y.
Copyright Year	2009
Description	Author affiliation: Hitachi, Ltd., Hitachi, Ltd., 1-280 Higashi-Koigakubo, Kokubunji, Tokyo 185-8601, Japan (Yasui, T.) \|\| Graduate School of Information Science and Technology, The University of Tokyo, The University of Tokyo, 7-3-1 Hongou, Bunkyo-ku, Tokyo, 113-8654 Japan (Ohta, K.; Ishikawa, Y.) \|\| Information Technology Center, The University of Tokyo, The University of Tokyo, 2-11-16 Yayoi, Bunkyo-ku, Tokyo, 113-8658 Japan (Hori, A.; Kamoshida, Y.; Matsuba, H.) \|\| Fujitsu Laboratories, Ltd., Fujitsu Laboratories, Ltd., 4-1-1, Kamikodanaka, Nakahara-ku, Kawasaki, Kanagawa, 211-8588 Japan (Sumimoto, S.)
Abstract	An on-demand file staging system, Catwalk, is proposed. Catwalk is designed so that it can run on any Linux clusters without any special or additional hardware. By having hook functions on the system calls of file operations, a file staging system can be transparent from the view of users, and users can be free from having wrong file staging scripts. In Catwalk, the file copying is done via normal TCP protocol so that Catwalk can run over ordinary, widely-used Ethernet. The stage-in file copy is pipelined to maximize the bandwidth from single file server. The performance of Catwalk is evaluated and compared with NFS using synthetic but realistic workloads. The evaluations show the stage-in performance with the pipeline technique is much better than the performance of NFS. The stage-out performance is comparable with the NFS performance despite the extra copying of files, and the file server is lightly loaded with the Catwalk stage-out while NFS entails much heavier server loads. The biggest problems of NFS are its centralized design and lack of scheduling for the parallel workloads. The performance of Catwalk shows that remote file access performance can be improved much better if file accesses are scheduled in a proper way. Thus the proposed file staging system can be a strong complement to NFS, especially for small clusters often having no dedicated parallel file system.
Starting Page	1
Ending Page	10
File Size	318039
Page Count	10
File Format	PDF
ISBN	9781424450114
ISSN	15525244
DOI	10.1109/CLUSTR.2009.5289189
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-08-31
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Linux File systems File servers Laboratories Processor scheduling Information technology Information science Hardware Protocols Ethernet networks parallel file system file staging cluster file system network file system
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Designing an enhanced PC cluster system for scalable network services

A Cost-Effective, High Bandwidth Server I/O network Architecture for Cluster Systems

Design and Implementation of Networking Virtualization for Cluster File Systems

A Jobs Allocation Strategy for Multiple DRBL Diskless Linux Clusters with Condor Schedulers

Design and implementation of PVFS-PM: a cluster file system on SCore

A SAMC-CDN Network System Based on Cluster Technology

Design and implementation of a PC-based video-on-demand system

A Information Monitoring and Job Scheduling System for Multiple Linux PC Clusters

A Novel Cluster Parallel File System

On-demand file staging system for Linux clusters

Similar Documents

Designing an enhanced PC cluster system for scalable network services

A Cost-Effective, High Bandwidth Server I/O network Architecture for Cluster Systems

Design and Implementation of Networking Virtualization for Cluster File Systems

A Jobs Allocation Strategy for Multiple DRBL Diskless Linux Clusters with Condor Schedulers

Design and implementation of PVFS-PM: a cluster file system on SCore

A SAMC-CDN Network System Based on Cluster Technology

Design and implementation of a PC-based video-on-demand system

A Information Monitoring and Job Scheduling System for Multiple Linux PC Clusters

A Novel Cluster Parallel File System

On-demand file staging system for Linux clusters