110 likes | 199 Views
Status of PIO library. Design goals. Provide parallel IO for all component models Reduce memory usage Potentially better performance Encapsulate complexity into library Simple interface for component developers to implement. Design goals (con’t). Extensible for future I/O technology
E N D
Design goals • Provide parallel IO for all component models • Reduce memory usage • Potentially better performance • Encapsulate complexity into library • Simple interface for component developers to implement
Design goals (con’t) • Extensible for future I/O technology • Backward compatible (node=0) • Support for binary(direct) and netcdf formats • Preserve format of input/output files • Support 2D and 3D arrays • Currently XY • Extensible to XZ or YZ
Terms • PnetCDF: [ANL] • High performance I/O • Different interface • Stable • netCDF4 + HDF5 [NCSA] • Same interface • Needs HDF5 library • Less stable • Lower performance • No support on Blue Gene
Terms (con’t) • Aggregators: • Number of IO tasks the MPI-IO subsystem will use • Accessible through hints • Not currently supported on BGL[?] • Processor stride: • Allows matching of subset of MPI IO nodes to system hardware
Component Model ‘issues’ • POP & CICE: • Missing blocks • Update of neighbors halo • Who writes missing blocks? • Asymmetry between read/write • ‘sub-block’ decompositions not rectangular • CLM • Decomposition not rectangular • Who writes missing data?
Some Concepts • IO decomp vs. COMP decomp • IO decomp == COMP decomp • MPI-IO + message aggregation • IO decomp != COMP decomp • Rearranger needed • MCT [Nov 2006] • ESMF [???] • No component specific info in library • Pair with existing communication tech • 1D arrays in library; component must flatten 2D and 3D arrays
What works • Binary I/O [direct] • Rearrange w/MCT + MPI-IO • MPI-IO no rearrangement • Test on POWER5 + BGL • Issues: • Some configurations fail • MCT errors • memory usage [ 277 vs 18 Mbytes/processor]
What works (con’t) • PnetCDF • MPI-IO no rearrangement • Some configurations work • Prototype added to POP2 • Reads restart and forcing files properly • POPIO benchmark • Test code for correctness and performance • Tested on 30K BGL processors in Oct 06 • Performance issues
Complexity • Mulitple ways to express decomposition • GDOF: global degree of freedom --> (MCT) • MPI datatypes: offsets + counts (MPI-IO) • Subarrays: start + count (pNetCDF) • Limited expressiveness • Will not support ‘sub-block’ in POP & CICE, CLM • Need common language for interface • Interface between component model and library