NDLI: Multi-resource fair sharing for datacenter jobs with placement constraints

Please wait, while we are loading the content...

Towards green aviation with python at petascale

Extreme-scale phase field simulations of coarsening dynamics on the sunway taihulight supercomputer

The vectorization of the tersoff multi-body potential: an exercise in performance portability

Development effort estimation in HPC

Scheduling-aware routing for supercomputers

PFEAST: a high performance sparse eigenvalue solver using distributed-memory linear solvers

Pinpointing scale-dependent integer overflow bugs in large-scale parallel applications

Simulation and performance analysis of the ECMWF tape library system

HARP: predictive transfer optimization based on historical analysis and real-time probing

Failure detection and propagation in HPC systems

Graph colouring as a challenge problem for dynamic graph processing on distributed systems

Understanding performance interference in next-generation HPC systems

Enhancing infiniband with openflow-style SDN capability

PIPES: a language and compiler for task-based programming on distributed-memory clusters

Granularity and the cost of error recovery in resilient AMR scientific applications

MUSA: a multi-level simulation approach for next-generation HPC machines

Exploring the potentials of parallel garbage collection in SSDs for enterprise storage systems

Translating OpenMP device constructs to OpenCL using unnecessary data transfer elimination

Optimizing memory efficiency for deep convolutional neural networks on GPUs

GreenLA: green linear algebra software for GPU-accelerated heterogeneous computing

Optimal execution of co-analysis for large-scale molecular dynamics simulations

A PCIe congestion-aware performance model for densely populated accelerator servers

Designing scalable b-Matching algorithms on distributed memory multiprocessors by approximation

An ephemeral burst-buffer file system for scientific applications

Distributed-memory large deformation diffeomorphic 3D image registration

Elastic multi-resource fairness: balancing fairness and efficiency in coupled CPU-GPU architectures

Characterizing parallel scientific applications on commodity clusters: an empirical study of a tapered fat-tree

Extended task queuing: active messages for heterogeneous systems

High-frequency nonlinear earthquake simulations on petascale heterogeneous supercomputers

Transient guarantees: maximizing the value of idle cloud capacity

Modeling dilute solutions using first-principles molecular dynamics: computing more than a million atoms with over a million cores

A highly effective global surface wave numerical simulation with ultra-high resolution

Increasing molecular dynamics simulation rates with an 8-fold increase in electrical power efficiency

MetaMorph: a library framework for interoperable kernels on multi- and many-core clusters

Evaluating HPC networks via simulation of parallel workloads

Block iterative methods and recycling for improved scalability of linear solvers

Compiler-directed lightweight checkpointing for fine-grained guaranteed soft error recovery

Real-time synthesis of compression algorithms for scientific data

SERF: efficient scheduling for fast deep neural network serving via judicious parallelism

Improving application resilience to memory errors with lightweight compression

An exploration of optimization algorithms for high performance tensor completion

Reliable and efficient performance monitoring in linux

Designing MPI library with on-demand paging (ODP) of infiniband: challenges and benefits

A domain-specific compiler for a parallel multiresolution adaptive numerical simulation environment

Extreme scale plasma turbulence simulations on top supercomputers worldwide

A machine learning framework for performance coverage analysis of proxy applications

Týr: blob storage meets built-in transactions

dCUDA: hardware supported overlap of computation and communication

Unprotected computing: a large-scale study of DRAM raw error rate on a supercomputer

Merge-based parallel sparse matrix-vector multiplication

Scalemine: scalable parallel frequent subgraph mining in a single large graph

Watch out for the bully!: job interference study on dragonfly network

A parallel algorithm for finding all pairs k-mismatch maximal common substrings

Server-side log data analytics for I/O workload characterization and coordination on large shared storage systems

ZNNi: maximizing the inference throughput of 3D convolutional networks on CPUs and GPUs

DCA: a DRAM-cache-aware DRAM controller

Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures

Perilla: metadata-based optimizations of an asynchronous runtime for adaptive mesh refinement

Refactoring and optimizing the community atmosphere model (CAM) on the sunway taihulight supercomputer

Multi-resource fair sharing for datacenter jobs with placement constraints

Simulations of below-ground dynamics of fungi: 1.184 pflops attained by automated generation and autotuning of temporal blocking codes

10M-core scalable fully-implicit solver for nonhydrostatic atmospheric dynamics

Enhanced MPSM3 for applications to quantum biological simulations

Truenorth ecosystem for brain-inspired computing: scalable systems, software, and applications

Flexfly: enabling a reconfigurable dragonfly through silicon photonics

Scalable non-blocking preconditioned conjugate gradient methods

Understanding error propagation in GPGPU applications

Performance modeling of in situ rendering

FlipBack: automatic targeted protection against silent data corruption

An efficient and scalable algorithmic method for generating large: scale random graphs

Evaluating and optimizing OpenCL kernels for high performance computing with FPGAs

The mont-blanc prototype: an alternative approach for HPC systems

Automating wavefront parallelization for sparse matrix computations

A parallel arbitrary-order accurate AMR algorithm for the scalar advection-diffusion equation

Caliper: performance introspection for HPC software stacks

DAOS and friends: a proposal for an exascale storage system

Daino: a high-level framework for parallel and efficient AMR on GPUs

A data driven scheduling approach for power management on HPC systems

Strassen's algorithm reloaded

Efficient delaunay tessellation through K-D tree decomposition

Measuring and understanding throughput of network topologies

Accelerating lattice QCD multigrid on GPUs using fine-grained parallelization

G-store: high-performance graph store for trillion-edge processing

High performance emulation of quantum circuits

Enabling efficient preemption for SIMT architectures with lightweight context switching

LIBXSMM: accelerating small matrix multiplications by runtime code generation

A multi-faceted approach to job placement for improved performance on extreme-scale systems

Multi-resource fair sharing for datacenter jobs with placement constraints

Content Provider	ACM Digital Library
Author	Li, Baochun Liang, Ben Wang, Wei Li, Jun
Abstract	Providing quality-of-service guarantees by means of fair sharing has never been more challenging in datacenters. Due to the heterogeneity of machine configurations, datacenter jobs frequently specify placement constraints, restricting them to run on a particular class of machines meeting specific hardware/software requirements. In addition, jobs have diverse demands across multiple resource types, and may saturate any of the CPU, memory, or storage resources. Despite the rich body of recent work on datacenter scheduling, it remains unclear how multi-resource fair sharing is defined and achieved for jobs with placement constraints. In this paper, we propose a new sharing policy called Task Share Fairness (TSF). With TSF, jobs are better off sharing the datacenter, and are better off reporting demands and constraints truthfully. We have prototyped TSF on Apache Mesos and confirmed its service guarantees in a 50-node EC2 cluster. Trace-driven simulations have further revealed that TSF speeds up 60% of tasks over existing fair schedulers.
Starting Page	1
Ending Page	12
Page Count	12
File Format	PDF
ISBN	9781467388153
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-11-13
Access Restriction	Subscribed
Subject Keyword	Placement constraints Cluster schedulers Multi-resource allocation Fairness
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in