The Python Interface to the Earth System Modeling Framework

1. The Python Interface to the Earth System Modeling Framework Ryan O’Kuinghttons Robert Oehmke Cecelia DeLuca Cooperative Institute for Research in Environmental Sciences NOAA Environmental Software Infrastructure and Interoperability Project American Meteorological Society Meeting Austin, Texas Jan 8, 2013

2. Introduction • The Earth System Modeling Framework (ESMF) is open source software for building modeling components, and coupling them together to form weather prediction, climate, coastal, and other applications. • Currently supports a full Fortran and limited C interface • ESMF provides a mature high performance regridding package • Transforms data from one grid to another by generating and applyinginterpolation weights • Supports structured and unstructured, global and regional, 2D and 3D grids, with many options • Fully parallel and highly scalable • The Python interface to ESMF (ESMPy) offers access to the regridding functionality in ESMF

3. Why does ESMF need a Python Interface? • Enables ESMF regridding to be used with very little effort, in an object oriented way: • Regridding applied as a callable Python object • Numpy array access to distributed data • Some users report computation times reduced from hours to minutes • Enables ESMF regridding to be used in other scientific packages with Python-based workflows – current users include: • UV-CDAT (PCMDI) – Ultrascale Visualization Climate Data Analysis Tools • PyFerret (NOAA) – Python based interactive visualization and analysis environment • Community Surface Dynamics Modeling System (CSDMS) – tools for hydrological and other surface modeling processes

4. ESMPy Classes • ESMF • Initialize and Finalize • Logging • Virtual Machine (parallel distribution) • Grid • Logically rectangular discretization object • Mesh • Unstructured mesh discretization object • Field • Grid or Mesh plus a data array and metadata • Derived type of the Numpy Array • Regrid • Callable object which operates on two Fields to compute and apply interpolation weights

5. A Few Words on Data Conventions… Following data conventions makes it easier to use data and tools. ESMPy grid files follow standard data file formats: • Climate and Forecast (CF) grid conventions • UGRID candidate CF convention for unstructured grids[1] • GridSpec CF convention for logically rectangular grids (accepted but not yet formally in CF) [2]

6. Supported Grids and Methods • Spherical coordinates – bilinear, higher order patch [3.4], or first order conservative regridding with: • Global or regional 2D logically rectangular grids • 2D unstructured meshes composed of triangles or quadrilaterals • Cartesian (x,y) coordinates: • Bilinear, higher order patch or first order conservative regridding between any pair of: • 2D meshes composed of triangles and quadrilaterals • 2D logically rectangular grids composed of a single patch • Bilinear and first order conservative regridding between any pair of: • 3D meshes composed of hexahedrons • 3D logically rectangular grids composed of a single patch 2D Unstructured Mesh From www.ngdc.noaa.gov Regional Grid FIM Unstructured Grid

7. Code Examples Create a Grid and access object information using Python properties: import esmpy, numpy grid = esmpy.Grid(maxIndex, numPeriDims=1, staggerlocs=[esmpy.StaggerLoc.CENTER]) lower_bounds = grid.lower_bounds[esmpy.StaggerLoc.CENTER] upper_bounds = grid.upper_bounds[esmpy.StaggerLoc.CENTER] Create a Mesh from UGRID formatted NetCDF file: mesh = esmpy.Mesh(“mesh_netcdf_file.nc”, esmpy.FileFormat.UGRID) Regridding object acting on two Fields: from esmpy import Regrid r1to2 = Regrid(grid, mesh, method=esmpy.RegridMethod.CONSERVATIVE) destination_field = r1to2(source_field) Write Grid and Mesh to .vtk formatted output files: grid.write(“source_grid”) mesh.write(“destination_mesh”) Field is derived from Numpy Array: exact_field = sqrt(source_field**5) exact_slice = exact_field[:,:,5:-1]

8. Getting Started Requirements: • Python 2.6 • Numpy1.6.1 (ctypes) • ESMF installation Testing: • Regression tested nightly on 5 platforms Supported Platforms: • Linux, Darwin, and Cray • Gfortran • OpenMPI Limitations: • No object for collections of Fields • No access to Mesh coordinates • No access to Field bounds

9. Status and Future Work • A prototype interface, ESMP, is currently available for download • The beta release of a new interface, ESMPy, is expected in early 2013 • More “pythonic” API • Later in 2013: addition of calendar and time management, and other classes from ESMF • Please contact esmf_support@list.woc.noaa.gov with questions or development requests Download http://earthsystemcog.org/projects/esmp/releases

10. Questions? References: 1. UGRID wiki: http://publicwiki.deltares.nl/display/NETCDF/Deltares+CF+proposal+for+Unstructured+Grid+data+model 2. GridSpec wiki: https://ice.txcorp.com/trac/modave/wiki/CFProposalGridspec 3. Khoei S.A., Gharehbaghi A. R., The superconvergent patch recovery technique and data transfer operators in 3d plasticity problems. Finite Elements in Analysis and Design, 43(8), 2007. 4. Hung K.C, Gu H., Zong Z., A modified superconvergent patch recovery method and its application to large deformation problems. Finite Elements in Analysis and Design, 40(5-6), 2004.

11. ESMP Interfacingwithctypes: _ESMF.ESMC_GridGetCoord.restype = ctypes.POINTER(ctypes.c_void_p) _ESMF.ESMC_GridGetCoord.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_uint, numpy.ctypeslib.ndpointer(dtype=numpy.int32), numpy.ctypeslib.ndpointer(dtype=numpy.int32), ctypes.POINTER(ctypes.c_int)] gridCoordPtr = _ESMF.ESMC_GridGetCoord(grid.struct.ptr, coordDim, staggerloc, exclusiveLBound, exclusiveUBound, ctypes.byref(lrc)) # adjustboundstobe 0 based exclusiveLBound = exclusiveLBound - 1 Allocating Numpy array buffers for memory allocated in ESMF: if grid.type == ESMP_TYPEKIND_R8: gridbuffer = numpy.core.multiarray.int_asbuffer( ctypes.addressof(gridCoordPtr.contents), numpy.dtype(numpy.float64).itemsize*size) Switching between Fortran and C array striding: Work in progress!