1 / 23

GNE: G lobal N EWS Modeling E nvironment

GNE Training Session 1. GNE: G lobal N EWS Modeling E nvironment. What it is / what it’s not How it works Components, samples Installation NEWS 2 Implementation. Emilio Mayorga May 2009 mayorga@apl.washington.edu http://www.marine.rutgers.edu/globalnews/GNE.

mora
Download Presentation

GNE: G lobal N EWS Modeling E nvironment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GNE Training Session 1 GNE: Global NEWS Modeling Environment What it is / what it’s not How it works Components, samples Installation NEWS 2 Implementation Emilio Mayorga May 2009 mayorga@apl.washington.edu http://www.marine.rutgers.edu/globalnews/GNE Content last edited August 28, 2008

  2. Generic GNE vs. MA Scenarios Specifics • Covering all details of Millennium Assessment (MA) Scenarios implementation is not an objective of these training sessions • But hands-on sessions and demonstrations will be based on MA 2000 (c00) scenario • Will use MA implementation to provide concrete examples

  3. GNE Training Goals • Introduce & describeGNE • Design goals, main features, code files • Describe a standard GNE model run • Major components, files (code, configuration, and input/output data), organization and conventions • Describe input pre-processing and output post-processing • Enable “model users” to set up and execute model scenario experiments • Modify input data sources used (especially for MA scenarios) • No Python coding needed Refer to GNE User Manual for more complete details. Sessions provide more interactive training and demonstration, while manual includes extensive details about syntax, usage, installation, etc.

  4. Training Sessions • Session 1. Introduction and Broad Overview This Presentation • Session 2. Running NEWS models with pre-existing, basin-aggregated input tables Interactive Session, in 2 parts (2 sub-sessions) • Part 1. Model run organization, configuration files, conventions • Part 2. Model execution; changing a model run (“experiments”) • Session 3. GIS pre-processing and model input/output post-processing Interactive Session

  5. Session 1 Content, Goals • Overall GNE design goals, highlights • Why Python? • Three GNE User Types • Training focus on “Model Users” (no Python coding needed) • GNE vs. end-to-end Global NEWS Run • What is covered by GNE vs. what is not • Major Components • Code, configuration files, input/output files, file organization and conventions • GNE Model Run folder & files • GNE Model Run • GIS Pre-Processing (GNE vs. other) • Post-Processing (GNE vs. other) • Python Installation and Base GNE Setup

  6. Overall GNE design goals, highlights • Dissolved models merged into a unified framework, rather than isolated sub-models; particulate models remain separate. • Focus on basin-scale calculations. • For users not needing to modify model equations, most interactions are handled via structured text configuration files. • Together with model equations in code, these files also serve as documentation of input files used and how output was created. • Complete model run can be maintained in a folder structure that includes all input data, model code, constants used, configuration files, and output tables. • Python/NumPy. Fairly easy to learn.Model equation code implemented in an intuitive, simple array syntax. • More complex GNE code handling input/output and overall model coordination hidden away and most users never have to see it. • Model run time (including GIS pre-processing) is short or trivial.

  7. Why Python? • Free, open source, high-level but general programming language with large user base in all fields of computing. Free = available to everyone. • Runs on all major operating systems. • Rapidly growing acceptance in science applications, including modeling & GIS. • Very clean, simplified syntax that makes it easy to learn, read existing code, and become productive quickly. • NumPy “add-on” used to code the model equations; intuitive, simple array syntax similar to Matlab. • Large library of default and optional add-ons contributed by users, for all sorts of applications. • Preferred language chosen by ESRI for data processing automation in ArcGIS (ArcGIS “GeoProcessing”).

  8. Three GNE User Types • Model Users.Setup or change a model run using a set of text configuration files; organize input and output files in a regular, structured fashion. Python skills not required. This training is focused on Model Users! • Model Developers.Focus on translating model equations into Python GNE code using simple code tools for variable management provided by the GNE framework. Largely isolated from the “behind-the-scenes”, low-level GNE functionality that handles file read-write and basic error checking. Should be fairly comfortable programming with Python & NumPy. • GNE Developers.Modify any aspect of GNE functionality, including model equations. Should be very comfortable programming with Python & NumPy.

  9. “Raw”, Heterogeneous GIS Source files Pre-processed, homogenized GIS Input files MNP Diffuse sources Diffuse sources Basin-scale Inputs tables Model Outputs Mass fluxes Manual steps, Ad-hoc automation GNE post- processing Direct Global NEWS inputs Global NEWS outputs MNP Point sources Point sources GNE GIS pre-processing Global NEWS model(s) Other post- processing Summaries, Analysis, Maps UNH Hydrology Hydrology GNE Complicated variables other than basin means/sums. Eg, dam retention, FQrem, lithology class GNE vs. end-to-end Global NEWS Run

  10. Major Components • Overall folder organization • Model run code, configuration, input/output files, etc. • Python Code • GNE “core” (file read/write, etc.) • Master controller (globalnews.py). • Science code (Global NEWS model): dissolved.py & particulate.py • Loads post-processor (postprocess.py) • Other, optional supporting files (batch runs, etc.) • Configuration files (text/ASCII) • Base folder paths (gensetup.cfg) • Dissolved calibration constants (constants.cfg) • GIS basin inputs pre-processing (gis2tbls.cfg) • Global NEWS model run (vars.cfg) • Loads post-processor (also vars.cfg)

  11. GNE Model Run Folder & Files Optional files (also in base folder, or in OTHERFILES) Base folder and sub-folders Files in base folder Also, documentation files.

  12. GNE Model Run Folder & Files:Python code Base folder and sub-folders Files in base folder Optional GIS pre-processing add-on Science (model) code Master controller Loads post-processor GNE Core code

  13. GNE Model Run Folder & Files:Configuration files Files in base folder Dissolved calibration constants GIS pre-processing Base folder paths Model run and loads post-processing

  14. GNE Model Run Folder & Files:Input and output files Base folder and sub-folders Basins, cell areas, and other basic input data (raster GIS and basin-scale tables) Homogenized raster GIS inputs Pre-processed basin-scale input tables Basin-scale model output and post-processed tables

  15. “Raw”, Heterogeneous GIS Source files Pre-processed, homogenized GIS Input files MNP Diffuse sources Diffuse sources Basin-scale Inputs tables Model Outputs Mass fluxes Manual steps, Ad-hoc automation GNE post- processing Direct Global NEWS inputs Global NEWS outputs MNP Point sources Point sources GNE GIS pre-processing Global NEWS model(s) Other post- processing Summaries, Analysis, Maps UNH Hydrology Hydrology GNE Complicated variables other than basin means/sums. Eg, dam retention, FQrem, lithology class GNE Model Run

  16. Sample GNE Model Run Configuration File vars.cfg

  17. Sample GNE Model Code calculate FEriv From dissolved.py

  18. “Raw”, Heterogeneous GIS Source files Pre-processed, homogenized GIS Input files MNP Diffuse sources Diffuse sources Basin-scale Inputs tables Model Outputs Mass fluxes Manual steps, Ad-hoc automation GNE post- processing Direct Global NEWS inputs Global NEWS outputs MNP Point sources Point sources GNE GIS pre-processing Global NEWS model(s) Other post- processing Summaries, Analysis, Maps UNH Hydrology Hydrology GNE Complicated variables other than basin means/sums. Eg, dam retention, FQrem, lithology class GNE GIS Pre-Processing

  19. “Raw”, Heterogeneous GIS Source files Pre-processed, homogenized GIS Input files MNP Diffuse sources Diffuse sources Basin-scale Inputs tables Model Outputs Mass fluxes Manual steps, Ad-hoc automation GNE post- processing Direct Global NEWS inputs Global NEWS outputs MNP Point sources Point sources GNE GIS pre-processing Global NEWS model(s) Other post- processing Summaries, Analysis, Maps UNH Hydrology Hydrology GNE Complicated variables other than basin means/sums. Eg, dam retention, FQrem, lithology class Other GIS Pre-Processing Mostly automated for MA scenarios, but in “ad-hoc” schemes, different software, etc.

  20. “Raw”, Heterogeneous GIS Source files Pre-processed, homogenized GIS Input files MNP Diffuse sources Diffuse sources Basin-scale Inputs tables Model Outputs Mass fluxes Manual steps, Ad-hoc automation GNE post- processing Direct Global NEWS inputs Global NEWS outputs MNP Point sources Point sources GNE GIS pre-processing Global NEWS model(s) Other post- processing Summaries, Analysis, Maps UNH Hydrology Hydrology GNE Complicated variables other than basin means/sums. Eg, dam retention, FQrem, lithology class GNE “Loads” Post-Processing Calculates basin input and output variables as loads

  21. “Raw”, Heterogeneous GIS Source files Pre-processed, homogenized GIS Input files MNP Diffuse sources Diffuse sources Basin-scale Inputs tables Model Outputs Mass fluxes Manual steps, Ad-hoc automation GNE post- processing Direct Global NEWS inputs Global NEWS outputs MNP Point sources Point sources GNE GIS pre-processing Global NEWS model(s) Other post- processing Summaries, Analysis, Maps UNH Hydrology Hydrology GNE Complicated variables other than basin means/sums. Eg, dam retention, FQrem, lithology class Other Post-Processing

  22. Python Installation and Base GNE Setup • Software Installation and Configuration • Is ArcGIS Installed? Potential conflicts with ArcGIS’ Python • Python (if ArcGIS not installed, or very old) • NumPy add-on • ArcGIS needed for GNE GIS pre-processing • (Other add-ons and custom scripts for additional post-processing: maps, regional summaries, etc.) • GNE • Copy GNE files (code, inputs, configuration files, folder structure). Unpack training zip file • Avoid blank spaces or characters other than underscores and alphanumeric characters anywhere in the folder path! • Testing • See User Manual for complete installation details, links to online Python/NumPy resources

  23. Next, in Session 2 … • Session 1. Introduction and Broad OverviewThis Presentation • Session 2. Running NEWS models with pre-existing, basin-aggregated input tables Interactive Session, in 2 parts (2 sub-sessions) • Part 1. Model run organization, configuration files, conventions • Part 2. Model execution; changing a model run (“experiments”) We will change and execute a few model runs • Session 3. GIS pre-processing and model input/output post-processing Interactive Session

More Related