NDLI: Timely offloading of result-data in HPC centers

Please wait, while we are loading the content...

Implementing Wilson-Dirac operator on the cell broadband engine

Accurate memory signatures and synthetic address traces for HPC applications

A freespace crossbar for multi-core processors

Advanced collective communication in aspen

CprFS: a user-level file system to support consistent file states for checkpoint and restart

Can software reliability outperform hardware reliability on high performance interconnects?: a case study with MPI over infiniband

Evaluating the effect of replacing CNK with linux on the compute-nodes of blue gene/l

Adaptive runtime tuning of parallel sparse matrix-vector multiplication on distributed memory systems

A compiler framework for optimization of affine loop nests for gpgpus

Optimizing irregular shared-memory applications for clusters

Orchestrating data transfer for the cell/B.E. processor

Exploiting idle register classes for fast spill destination

Automatic analysis of speedup of MPI applications

Many-core GPU computing with NVIDIA CUDA

Biomedical image analysis on a cooperative cluster of GPUs and multicores

Preserving time in large-scale communication traces

An approach for adaptive DRAM temperature and power management

The deep computing messaging framework: generalized scalable message passing on the blue gene/P supercomputer

Timely offloading of result-data in HPC centers

Soft error vulnerability of iterative linear algebra methods

Power-aware dynamic placement of HPC applications

Fast scan algorithms on graphics processors

Rotating register allocation with multiple rotating branches

Performance portable optimizations for loops containing communication operations

CUBA: an architecture for efficient CPU/co-processor data communication

Analysis of dynamic power management on multi-core processors

Analyzing memory access intensity in parallel programs on multicore

Challenges on the road to exascale computing

Data mining on the cell broadband engine

The shared-thread multiprocessor

A projection-based optimization framework for abstractions with application to the unstructured mesh domain

Shifted declustering: a placement-ideal layout scheme for multi-way replication storage architecture

Autonomous learning for efficient resource utilization of dynamic VM migration

Three-dimensional delaunay refinement for multi-core processors

Automatic SIMD vectorization of chains of recurrences

Phasers: a unified deadlock-free construct for collective and point-to-point synchronization

Efficient computation of sum-products on GPUs through software-managed cache

Focused prefetching: performance oriented prefetching based on commit stalls

A regression-based approach to scalability prediction

Petaflop/s, seriously

Timely offloading of result-data in HPC centers

Content Provider	ACM Digital Library
Author	Vazhkudai, Sudharshan S. Butt, Ali R. Monti, Henry M.
Abstract	High performance computing is facing an exponential growth in job output dataset sizes. This implies a significant commitment of supercomputing center resources---most notably, precious scratch space---in handling data staging and offloading. However, the scratch area is typically managed using simple "purge policies", without sophisticated "end-user data services" that are required to balance center's resource consumption and user serviceability. End-user data services such as offloading are performed using point-to-point transfers that are unable to reconcile center's purge and users delivery deadlines, unable to adapt to changing dynamics in the end-to-end data path and are not fault-tolerant. We propose a robust framework for the timely, decentralized offload of result data, addressing the aforementioned significant gaps in extant direct-transfer-based offloading. The decentralized offload is achieved using an overlay of user-specified intermediate nodes and well known landmark nodes. These nodes serve as a means both to provide multiple data-flow paths, thereby maximizing bandwidth as well as provide fail-over capabilities for the offload. We have implemented our techniques within a production job scheduler (PBS) and data transfer tool (BitTorrent), and our evaluation shows that the offloading times can be significantly reduced (90.2% for a 2.1 GB file), while also meeting center-user Service Level Agreements.
Starting Page	124
Ending Page	133
Page Count	10
File Format	PDF
ISBN	9781605581583
DOI	10.1145/1375527.1375548
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2008-06-07
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Hpc center serviceability High performance data management Offloading End-user data delivery
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in