Loading...
Please wait, while we are loading the content...
Similar Documents
Collective I / O Optimizations for Adaptive Mesh Refinement Data Writes on Lustre File System
| Content Provider | Semantic Scholar |
|---|---|
| Author | Devendran, Dharshi Byna, Suren Dong, Bin Straalen, Brian Van Johansen, Hans Keen, Noel Samatova, Nagiza F. |
| Copyright Year | 2016 |
| Abstract | Adaptive mesh refinement (AMR) applications refine small regions of a physical space. As a result, when AMR data has to be stored in a file, writing data involves storing a large number of small blocks of data. Chombo is an AMR software library for solving partial differential equations over block-structured grids, and is used in large-scale climate and fluid dynamics simulations. Chombo’s current implementation for writing data on an AMR hierarchy uses several independent write operations, causing low I/O performance. In this paper, we investigate collective I/O optimizations for Chombo’s write function. We introduce Aggregated Collective Buffering (ACB) to reduce the number of small writes. We demonstrate that our approach outperforms the current implementation by 2× to 9.1× and the MPI-IO collective buffering by 1.5× to 3.4× on the Edison and Cori platforms at NERSC using the ChomboIO benchmark. We also test ACB on the BISICLES Antarctica benchmark on Edison, and show that it outperforms the current implementation by 13.1× to 20×, and the MPI-IO collective buffering by 6.4× to 12.8×. Using the Darshan I/O characterization tool, we show that ACB makes larger contiguous writes than collective buffering at the POSIX level, and this difference gives ACB a significant performance benefit over collective buffering. |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | https://cug.org/proceedings/cug2016_proceedings/includes/files/pap148s2-file1.pdf |
| Alternate Webpage(s) | https://sdm.lbl.gov/~sbyna/research/papers/201605_CUG2016_ACB.pdf |
| Alternate Webpage(s) | https://cug.org/proceedings/cug2016_proceedings/includes/files/pap148s2-file2.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
