Naming, Storage, and Representation of WRF Metadata Attributes in myLEAD

1. Naming, Storage, and Representation of WRF Metadata Attributes in myLEAD Sangmi Lee Pallickara, Scott Jensen and Beth Plale Indiana university Thanks to Suresh Marru, Dan Weber, and Kevin Thomas for their effort and time in discussions about the input parameters for the model run

2. Storing Metadata: how and when • An XML document containing metadata for an experiment (e.g., WRF parameters) is delivered to the myLEAD server, parsed, and the metadata is stored in a relational database. • The actual setup files, data files, and derived result files are stored in a file system (SRB). Only metadata about the files is stored in myLEAD. • As users create experiments through the portal interface, information about the experiment is delivered to myLEAD at each step (e.g. which project, which experiment, which workflow template, etc.). • As the workflow runs, application-specific metadata is delivered to myLEAD, (e.g. which workflow template, instance, WRF parameters, etc.) as well as metadata regarding each file used or generated.

3. Data Model: how metadata is stored • Collection table • Projects/experiments/collections are stored in the collection table. The hierarchical Parent/Child relationships are recorded in the Lead Parent table. • Attributes • The metadata about collections and files are stored as attributes under the particular collection or file. • Static/dynamic attributes.

4. Example • Create a new project : Project 2005 • Create a new experiment : ExpAugust2005 • Add a workflow template (template1) from existing workflow templates • Add a collection input_Data View from myWorkspace portal In backend RDBMS (simplified) Collections Lead parent Project2005 My Workspace ExpAugust2005 Input_Data Project2005 Collection attributes ExpAugust2005 template1 template1 Input_Data Attribute Definition Collection Blob Elem Actual data of template1

5. Project: MyProject Experiment: My WRF Run Attribute: Run Name Element: Run Name Attribute: Grid Setup Element: Horizontal Grid Space Element: Vertical Grid Space Parent-Child Relationship Element: Latitude of Domain Center Parent Parent Child Child Parent ID Type Level Child ID Type Level Element: Longitude of Domain Center 1 7 1 2 6 2 Users Creator ID DN Name 1 Kelvin Droegemeier /C=US/O=National Center for Supercomputing Applications/CN=Kelvin Droegemeier /C=US/O=National Center for Supercomputing Applications/CN=Dan Weber 2 Dan Weber 3 /C=US/O=National Center for Supercomputing Applications/CN=Suresh Marru Suresh Marru Collections Collection Attributes Attribute Definitions Creator Col Col ID Attr ID ID Type Col ID Name Attr ID Name 2 1 1 2 7 MyProject 2 2 1 Run Name 2 2 6 My WRF Run 2 Grid Setup Collection String Elements Col ID Attr ID Elem ID Element Value 2 1 1 Run001 Element Definitions Elem Elem ID Attr ID Type Name Type 6 1 1 6 Run Name Collection Float Elements is String 2 2 5 Horizontal Grid Space Col ID Attr ID Elem ID Element Value 3 2 5 Vertical Grid Space 2 2 2 1000.00 Type 5 4 2 5 Latitude of Domain Center 2 2 3 500.00 is Float 5 2 5 Longitude of Domain Center 2 2 4 36.00 2 2 5 -100.00 Note: Elements within an attribute can be of different types. Example – Adding Input Parameters as attributes

6. What’s the purpose of attributes? • We can’t always remember the name of a file we used last year. Likewise, we may not remember the name of an experiment that we ran last year or even last month. • Attributes allow us to find experiments, files, or collections of files based on properties of the data, model, or derived results. • Attributes can be recorded for any level in the structure of an experiment. Files, collections, experiments, and even projects can have attributes. • Attributes provide the key to query functionality. • Files can be searched by the attributes and their values • Experiments can be searched by attributes and their values • Attributes provide better sharing of data, models, and results.

7. Search Experiment by the model parameters(1)

8. Search Experiment by the input parameters(2)

9. Storing WRF input parameters as attributes in myLEAD • WRF/ARPS input parameters are stored as attributes of experiment. • Users can search for experiment based on input/model parameters (model run name, Grid etc.). • Currently we have added the namelist parameters that the MET group identified as the most common, but Kevin is looking at the ARPS namelist file to determine additional parameters that should be recorded.

10. ARPS & WRF input parameters • Weather Models use FORTRAN NAMELIST to specify input to the Fortran applications and change the way they execute without having to recompile.

11. Catagorizing input parameters* • Category I: Cross cutting - Mandatory user editable - • Category II: File Handling - Mandatory workflow editable • Category III: Service specific - Optional (can be defaulted for most cases.) This number changes for different group of users • Category IV: MPI Parameters - workflow editable • Category V: Never Changing parameters * Thanks to Suresh

12. Category I- Description • Every User has to enter these values at the start of the experiment. • These input parameters are common among most of the services of ARPS and WRF systems. • These parameters have to be maintained consisting in all the services. • These parameters have to parsed by the portal as they determine the magnitude of resources needed.

13. Category I - Sub-Categories • Model Run Name • Model Dimension Parameters • Model Grid Setup Parameters • Model Initialization Parameters • Map Projection Parameters • Time Integration Control Parameters

14. Category I Variables List(for quick reference the namelist block which contains these variables in the of arps.input file is mentioned, all these variables are explained in the following slides) • runname - &jobname • nx, ny, nz, - &grid_dims • dx,dy,dz, ctrlat, ctrlon - &grid • initime - &initialization

15. Model Run Name • Variable runname - this has to be obtained from user and first 6 characters have to be prefixed to intermediate and output logical file names

16. Model Dimension Parameters • Number of Grid Points Identifies the physical dimension of the model domain Variables nx, ny & nz nx and ny are used in inputs to arpstrn, wrfsi, ext2arps, 88d2arps, nids2arps, mci2arps, adas, arps2wrf, wrf, wrf2arps, arpsplt & arpsverif nz used in ext2arps, 88d2arps, nids2arps, adas, arps2wrf, wrf, wrf2arps, arpsplt (nx and ny determine the number of processors needed and how they need to be split)

17. Number of Grid Points in X, Y direction • nx, ny- grid points in Y and Y direction ( This input has to be collected from user and +3 has to be added when setting nx and ny in adas input files and +1 when setting to wrf input files, because over the forecast grid, arps system uses 3 fake points and wrf uses 1 fake point)

18. Number of Grid Points in Z direction • nz- grid points in z direction ( No fake points need to be added to nz and also nz need not considered for determining number of processors in mpi runs)

19. Model Grid Setup Parameters • Identifies the center of the model domain and grid spacing spacing in x, y and z directions. • Variables dx, dy, dz, ctrlat, ctrlon • dx and dy must be set in arpstrn, wrfsi, ext2arps, 88d2arps, nids2arps, mci2arps, adas, arps2wrf, wrf & wrf2arps • dz must be set in ext2arps, 88d2arps, nids2arps, adas, arps2wrf, wrf, & wrf2arps • ctrlat and ctrlon must be set in arpstrn, wrfsi, ext2arps, 88d2arps, nids2arps, mci2arps

20. Grid Spacing • dx, dy, dy denote the grid spacing in X, Y and Z directions respectively. • dx, dy and dz are specified in meters. • dx and dy should always be the same • dz should only be exposed to advanced users, for others this should be set to a value based on nz.

21. Model Initialization Parameters • initime - the model initilization time specified as character string in the UTC format - 'yyyy-mn-dd.hh:mm:ss'