Loading...
Please wait, while we are loading the content...
Similar Documents
Damaris: Efficiently Leveraging I/O Cores for Scalable Post-Petascale HPC Simulations
| Content Provider | Semantic Scholar |
|---|---|
| Author | Dorier, Matthieu |
| Copyright Year | 2012 |
| Abstract | Many science domains, such as Earth science, life science, physics, and chemistry, rely largely on computationally intensive simulations. Such simulations generate tremendous quantities of information. Conventional practice consists in storing data on disk, moving it off-site, reading it into a workflow, and analyzing it. This becomes increasingly harder because of the large data volumes generated at fast rates, in contrast to a slower increase in the performance of storage systems. Rapidly storing this data, protecting it from loss, and analyzing it to understand the results are significant challenges. Typically, I/O is periodically and concurrently performed by all processes, which leads to I/O bursts. On current petascale systems, resource contention and substantial variability of I/O performance significantly impact both the overall application performance and its predictability. As the community considers designs for exascale systems, there is a growing consensus in the international community that revolutionary new approaches are needed in computational science storage. Motivated by these challenges in the context of the NCSA’s Blue Waters supercomputer project [16], we propose Damaris, an approach that efficiently leverages a subset of cores on each multicore SMP node to act as a data management service. Damaris has been implemented as an open-source middleware. As opposed to other “space-partitioning” approaches, Damaris makes an efficient use of intra-node shared memory to avoid costly copies or movements of data and provides a plugin system to bridge simulations with any I/O or data analysis library. It efficiently removes I/O related costs and is able to gather, compress and store data in an overhead-free, asynchronous manner for subsequent offline analysis. We evaluate our approach on up to 9216 cores on the Kraken Cray XT5 supercomputer [17], with the CM1 atmospheric model, one of the target HPC applications for the Blue Waters project. By overlapping I/O with computation and by gathering data into large files while avoiding synchronization between cores, our solution brings several benefits: |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | https://src.acm.org/binaries/content/assets/src/2011/matthieudorier.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
