NDLI: Checkpointing strategies for parallel jobs

Please wait, while we are loading the content...

First-principles calculations of electron states of a silicon nanowire with 100,000 atoms on the K computer

Optimizing symmetric dense matrix-vector multiplication on GPUs

Liszt: a domain specific language for building portable mesh-based PDE solvers

CudaDMA: optimizing GPU memory bandwidth via warp specialization

Parallel random numbers: as easy as 1, 2, 3

Server-side I/O coordination for parallel file systems

GreenSlot: scheduling energy consumption in green datacenters

Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems

The IBM Blue Gene/Q interconnection network and message unit

I/O streaming evaluation of batch queries for data-intensive computational turbulence

FTI: high performance fault tolerance interface for hybrid systems

Fast implementation of DGEMM on Fermi GPU

Virtual I/O caching: dynamic storage cache management for concurrent workloads

Scalable hashing for shared memory supercomputers

Evaluating the viability of process replication reliability for exascale systems

TRACON: interference-aware scheduling for data-intensive applications in virtualized environments

Large scale debugging of parallel tasks with AutomaDeD

Sniper: exploring the level of abstraction for scalable and accurate parallel multi-core simulation

Performance of the community earth system model

Hadoop acceleration through network levitated merge

Copernicus: a new paradigm for parallel adaptive molecular dynamics

A scalable eigensolver for large scale-free graphs using 2D graph partitioning

SciHadoop: array-based query processing in Hadoop

High-performance lattice QCD for multi-core based parallel systems using a cache-friendly hybrid threaded-MPI approach

Scalable implementations of accurate excited-state coupled cluster theories: application of high-level methods to porphyrin-based systems

Optimizing the Barnes-Hut algorithm in UPC

SC 2011 Keynote

Atomistic nanoelectronic device engineering with sustained performances up to 1.44 PFlop/s

Tiled QR factorization algorithms

Simplified parallel domain traversal

Dymaxion: optimizing memory access patterns for heterogeneous systems

QoS support for end users of I/O-intensive applications using shared storage systems

A 'cool' load balancer for parallel applications

Unitary qubit lattice simulations of multiscale phenomena in quantum turbulence

High-efficiency server design

Parallel index and query for large scale data analysis

Checkpointing strategies for parallel jobs

Scalable fast multipole methods on distributed heterogeneous architectures

SCMFS: a file system for storage class memory

An early performance analysis of POWER7-IH HPC systems

Modeling and tolerating heterogeneous failures in large parallel systems

Flexible resource allocation for reliable virtual cluster computing systems

Efficient data race detection for distributed memory parallel programs

MAximum Multicore POwer (MAMPO): an automatic multithreaded synthetic power virus generation framework for multicore systems

Extracting ultra-scale Lattice Boltzmann performance via hierarchical and distributed auto-tuning

Purlieus: locality-aware resource allocation for MapReduce in a cloud

Enabling and scaling biomolecular simulations of 100 million atoms on petascale machines with a multicore-optimized message-driven runtime

Scalable stochastic optimization of complex energy systems

On the duality of data-intensive file system design: reconciling HDFS and PVFS

Scaling lattice QCD beyond 100 GPUs

Hardware/software co-design for energy-efficient seismic modeling

Avoiding hot-spots on two-level direct networks

Analyst Crossfire

Peta-scale phase-field simulation for dendritic solidification on the TSUBAME 2.0 supercomputer

Parallel reduction to condensed forms for symmetric eigenvalue problems using aggregated fine-grained and memory-aware kernels

Physis: an implicitly parallel programming model for stencil computations on large-scale GPU-accelerated supercomputers

GROPHECY: GPU performance projection from CPU code skeletons

Topology-aware data movement and staging for I/O acceleration on Blue Gene/P supercomputing systems

Reducing electricity cost through virtual machine placement in high performance computing clouds

An image compositing solution at scale

Using the TOP500 to trace and project technology and architecture trends

ISABELA-QA: query-driven analytics with ISABELA-compressed extreme-scale scientific data

BlobCR: efficient checkpoint-restart for HPC applications on IaaS clouds using virtual disk image snapshots

Multi-science applications with single codebase - GAMER - for massively parallel architectures

Optimized pre-copy live migration for memory intensive applications

A similarity measure for time, frequency, and dependencies in large-scale workloads

System implications of memory reliability in exascale computing

Auto-scaling to minimize cost and meet application deadlines in cloud workflows

Multithreaded Global Address Space Communication Techniques for Gyrokinetic Fusion Applications on Ultra-Scale Platforms

Highly scalable ab initio genomic motif identification

A distributed look-up architecture for text mining applications using MapReduce

Parallelization design on multi-core platforms in density matrix renormalization group toward 2-D quantum strongly-correlated systems

Parallel breadth-first search on distributed memory systems

End-to-end network QoS via scheduling of flexible resource reservation requests

Large scale plane wave pseudopotential density functional theory calculations on GPU clusters

A fast solver for modeling the evolution of virus populations

Improving communication performance in dense linear algebra via topology aware collectives

Petaflop biofluidics simulations on a two million-core system

A new computational paradigm in multiscale simulations: application to brain blood flow

Checkpointing strategies for parallel jobs

Content Provider	ACM Digital Library
Author	Vivien, Frédéric Bougeret, Marin Robert, Yves Rabie, Mikael Casanova, Henri
Abstract	This work provides an analysis of checkpointing strategies for minimizing expected job execution times in an environment that is subject to processor failures. In the case of both sequential and parallel jobs, we give the optimal solution for exponentially distributed failure inter-arrival times, which, to the best of our knowledge, is the first rigorous proof that periodic checkpointing is optimal. For non-exponentially distributed failures, we develop a dynamic programming algorithm to maximize the amount of work completed before the next failure, which provides a good heuristic for minimizing the expected execution time. Our work considers various models of job parallelism and of parallel checkpointing overhead. We first perform extensive simulation experiments assuming that failures follow Exponential or Weibull distributions, the latter being more representative of real-world systems. The obtained results not only corroborate our theoretical findings, but also show that our dynamic programming algorithm significantly outperforms previously proposed solutions in the case of Weibull failures. We then discuss results from simulation experiments that use failure logs from production clusters. These results confirm that our dynamic programming algorithm significantly outperforms existing solutions for real-world clusters.
Starting Page	1
Ending Page	11
Page Count	11
File Format	PDF
ISBN	9781450307710
DOI	10.1145/2063384.2063428
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2011-11-12
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Checkpointing Sequential job Fault-tolerance Parallel job
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in