NDLI: SRC: Damaris - using dedicated i/o cores for scalable post-petascale HPC simulations

Please wait, while we are loading the content...

Rethinking shared-memory languages and hardware

An execution strategy and optimized runtime support for parallelizing irregular reductions on modern GPUs

Transactional conflict decoupling and value prediction

Scalable fine-grained call path tracing

Page placement in hybrid memory systems

SecureME: a hardware-software approach to full system security

Coordinating processor and main memory for efficientserver power control

Challenges and opportunities in renewable energy and energy efficiency

Characterizing the impact of soft errors on iterative methods in scientific computing

Optimizing the datacenter for data-centric workloads

Performance modeling as the key to extreme scale computing

Mint: realizing CUDA performance in 3D stencil methods with annotated C

Automating GPU computing in MATLAB

Cost-effectively offering private buffers in SoCs and CMPs

Cosmic microwave background map-making at the petascale and beyond

The elephant and the mice: the role of non-strict fine-grain synchronization for modern many-core architectures

SRC: information retrieval as a persistent parallel service on supercomputer infrastructure

Automatic generation of executable communication specifications from parallel applications

Multiset signatures for transactional memory

Generic topology mapping strategies for large-scale parallel architectures

Performance impact and interplay of SSD parallelism through advanced commands, allocation strategy and data granularity

Processing data streams with hard real-time constraints on heterogeneous systems

Optimizing throughput/power trade-offs in hardware transactional memory using DVFS and intelligent scheduling

High performance linpack benchmark: a fault tolerant implementation without checkpointing

Predictive coordination of multiple on-chip resources for chip multiprocessors

MDR: performance model driven runtime for heterogeneous parallel platforms

Using GPUs to compute large out-of-card FFTs

A composite and scalable cache coherence protocol for large scale CMPs

A QHD-capable parallel H.264 decoder

$F^{2}BFLY:$ an on-chip free-space optical network with wavelength-switching

SRC: Damaris - using dedicated i/o cores for scalable post-petascale HPC simulations

Hystor: making the best use of solid state drives in high performance storage systems

ZEBRA: a data-centric, hybrid-policy hardware transactional memory design

Modeling the performance of an algebraic multigrid cycle on HPC platforms

An idiom-finding tool for increasing productivity of accelerators

Active pebbles: parallel programming for data-driven applications

Automatic SIMD vectorization of fast fourier transforms for the larrabee and AVX instruction sets

Controlling cache utilization of HPC applications

MP-PIPE: a massively parallel protein-protein interaction prediction engine

Karma: scalable deterministic record-replay

SRC: FenixOS - a research operating system focused on high scalability and reliability

SRC: soft error detection and recovery for high performance linpack

Poster: DVFS management in real-processors

SRC: OpenSHMEM library development

SRC: enabling petascale data analysis for scientific applications through data reorganization

SRC: virtual i/o caching: dynamic storage cache management for concurrent workloads

SRC: an automatic code overlaying technique for multicores with explicitly-managed memory hierarchies

Poster: programming clusters of GPUs with OMPSs

Poster: revisiting virtual channel memory for performance and fairness on multi-core architecture

Poster: implications of merging phases on scalability of multi-core architectures

SRC: facilitating efficient parallelization of information storage and retrieval on large data sets

SRC: automatic extraction of SST/macro skeleton models

SRC: Damaris - using dedicated i/o cores for scalable post-petascale HPC simulations

Content Provider	ACM Digital Library
Author	Dorier, Matthieu
Abstract	As we enter the post-petascale era, scientific applications running on large-scale platforms generate increasingly larger amounts of data for checkpointing or offline visualization, which puts current storage systems under heavy pressure. Unfortunately, I/O scalability rapidly fades behind the increasing computation power available, and thereby reduced the overall application performance scalability. We consider the common case of large-scale simulations who alternate between computation phases and I/O phases. Two main approaches have been used to handle these I/O phases: 1) each process writes an individual file, leading to a very large number of files from which it is hard to retrieve scientific insights; 2) processes synchronize and use collective I/O to write to the same shared file. In both cases, because of mandatory communications betweens processes during the computation phase, all processes enter the I/O phase at the same time, which leads to huge access contention and extreme performance variability. Previous research efforts have focused on improving each layer of the I/O stack separately: at the highest level scientific data formats like HDF5 allow to keep a high degree of semantics within files, while leveraging MPI-IO optimizations. Parallel file systems like GPFS or PVFS are also subject to optimization efforts, as they usually represent the main bottleneck of this I/O stack. As a step forward, we introduce Damaris (Dedicated Adaptable Middleware for Application Resources Inline Steering), an approach targeting large-scale multicore SMP supercomputers. The main idea is to dedicate one or a few cores on each node to I/O and data processing to provide an efficient, scalable-by-design, in-compute-node data processing service. Damaris takes into account user-provided information related to the application, the file system and the intended use of the datasets to better schedule data transfers and processing. It may also respond to visualization tools to allow in-situ visualization without impacting the simulation. We tested our implementation of Damaris as an I/O backend for the CM1 atmospheric model, one of the application intended to run on next generation supercomputer BlueWaters at NCSA. CM1 is a typical MPI application, originally writing one file per process at each checkpoint using HDF5. Deployed on 1024 cores on BluePrint, the BlueWater's interim system at NCSA with GPFS as underlying filesystem, this approach induces up to 10 seconds overhead in checkpointing phases every 2 minutes, with a high variability in the time spent by each process to write its data (from 1 to 10 seconds). Using one dedicated I/O core in each 16-cores SMP node, we completely remove this overhead. Moreover, the time spared by the I/O core enables a better compression level, thus reducing both the number of files produced (by a factor of 16) and the total data size. Experiments conducted on the French Grid5000 testbed with PVFS as underlying filesystem and a 24 cores/node cluster emphasized the benefit of our approach, which allows communication and computation to overlap, in a context involving high network contention at multiple levels.
Starting Page	370
Ending Page	370
Page Count	1
File Format	PDF
ISBN	9781450301022
DOI	10.1145/1995896.1995953
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2011-05-31
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Dedicated cores Multicore architectures Exascale computing I/o
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Damaris: Efficiently Leveraging I/O Cores for Scalable Post-Petascale HPC Simulations

Damaris: How to Efficiently Leverage Multicore Parallelism to Achieve Scalable, Jitter-free I/O

it Damaris: Leveraging Multicore Parallelism to Mask I/O Jitter

On the benefit of dedicating cores to mask I/O jitter in HPC simulations

Damaris/Viz: A nonintrusive, adaptable and user-friendly in situ visualization framework

Damaris: Addressing Performance Variability in Data Management for Post-Petascale Simulations

On the energy footprint of I/O management in Exascale HPC systems

Addressing the Challenges of I/O Variability in Post-Petascale HPC Simulations

A performance and energy analysis of I/O management approaches for exascale systems

SRC: Damaris - using dedicated i/o cores for scalable post-petascale HPC simulations

Similar Documents

Damaris: Efficiently Leveraging I/O Cores for Scalable Post-Petascale HPC Simulations

Damaris: How to Efficiently Leverage Multicore Parallelism to Achieve Scalable, Jitter-free I/O

it Damaris: Leveraging Multicore Parallelism to Mask I/O Jitter

On the benefit of dedicating cores to mask I/O jitter in HPC simulations

Damaris/Viz: A nonintrusive, adaptable and user-friendly in situ visualization framework

Damaris: Addressing Performance Variability in Data Management for Post-Petascale Simulations

On the energy footprint of I/O management in Exascale HPC systems

Addressing the Challenges of I/O Variability in Post-Petascale HPC Simulations

A performance and energy analysis of I/O management approaches for exascale systems

SRC: Damaris - using dedicated i/o cores for scalable post-petascale HPC simulations