1 / 28

ESMF Model Outputs Data Access Framework: Search, Browse, Compare, Analyze

This framework allows users to search, browse, compare, validate, and analyze ESMF model outputs, facilitating data access and insight.

javierdavis
Download Presentation

ESMF Model Outputs Data Access Framework: Search, Browse, Compare, Analyze

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Data Access Framework for ESMF Model Outputs Roland Schweitzer Steve Hankin Jonathan Callahan Kevin O’Brien Ansley Manke

  2. Some Motivation Model Components produce …  Model Runs which produce …  Data and Metadata which should/could produce …  A data portal which allows user to: • Search, browse, compare, validate and analyze results …  which in turn helps produce insight 2005 ESMF Workshop

  3. Some More Motivation • ESMF and the ESCurator manage components, runs and output • LAS is a candidate to provide a data portal 2005 ESMF Workshop

  4. The Live Access Server (LAS) A highly configurable Web server designed to provide flexible access to geo-referenced scientific data 2005 ESMF Workshop

  5. LAS is an information broker that: ingests descriptions of data data sets variables (optional reshuffling of above into new categories) domains: 1d-4d special constraints (ensemble index) rendering hints (smoothing, palettes) presents the descriptions to the user in a coherent fashion asks other software to perform tasks for the user and gives the results to the user LAS 2005 ESMF Workshop

  6. Architecture Components User Interface Server (UIS) Product Server (PS) Backend Server (BES) Backend Server (BES) Ferret Data (OPeNDAP) Server (FDS) Data 2005 ESMF Workshop

  7. LAS Offers • Independence from underlying data • Uniform interface for browse (visualize), compare, validate and analyze results for many data types • Easy way to make connections to metadata and documentation • Customizable user interface 2005 ESMF Workshop

  8. Example (Multiple Hierarchies) 2005 ESMF Workshop

  9. Example (Collapse Hierarchies) Experimental Development Server 2005 ESMF Workshop

  10. Example (Custom Headers) The NVODS Server (http://www.ferret.noaa.gov/nvods) 2005 ESMF Workshop

  11. Example (Adopt a Drifter) 2005 ESMF Workshop

  12. Return to the Motivation Model Components produce …  Model Runs which produce …  Data and Metadata which should/could produce …  A data portal which allows user to: • Search, browse, compare, validate and analyze results …  which in turn helps produce insight 2005 ESMF Workshop

  13. Data Browsing 2005 ESMF Workshop

  14. Analysis and Comparison • Compute differences • Can define new variables that are the result of some analysis operation (done in delayed mode so only data that is needed is computed) 2005 ESMF Workshop

  15. Analysis and Comparison • Can use FDS to define regriddings (or other analysis) to allow more complex comparisons 2005 ESMF Workshop

  16. Validation • Use FDS to create samplings of in situ observations which can be compared to model data • (This is a planned for a future release.) 2005 ESMF Workshop

  17. Detailed Analysis • Can jump to local tool (Matlab, IDL, Ferret, etc.) by requesting scripts for that application • Scripts encode access to remote LAS data via FDS • Can take advantage of FDS analysis to compute some data reductions and transfer only the result to the local application 2005 ESMF Workshop

  18. Additional Flexibility beyond Configuration • Use LAS framework with different backend engine (new Product Server is designed to make this as easy as possible) • Web-services interfaces • To Product Server from other clients • Will allow implementation of “well-known” service like WMS 2005 ESMF Workshop

  19. Architecture Components User Interface Server (UIS) Product Server (PS) Backend Server (BES) Backend Server (BES) Ferret Data (OPeNDAP) Server (FDS) Data 2005 ESMF Workshop

  20. Access Control and the GRID • The new architecture makes participating as a GRID partner easier • Web services-based authentication schemes can be used by the Product Server and Backend Server to get authentication information (if not supplied by a browser certificate). • Implemented as another “filter” in a already-existing mechanism for filtering on IP-address, number of accesses, etc. • Configuration metadata can be augmented to identify allowed access “roles” by dataset and/or variable (and maybe by output product). 2005 ESMF Workshop

  21. Return to the Motivation Model Components produce …  Model Runs which produce …  Data and Metadata which should/could produce …  A data portal which allows user to: • Search, browse, compare, validate and analyze results …  which in turn helps produce insight 2005 ESMF Workshop

  22. More Information • www.ferret.noaa.gov • Subscribe to the LAS User’s mailing list 2005 ESMF Workshop

  23. The End 2005 ESMF Workshop

  24. OPeNDAP Server FDS LAS Architecture with OPeNDAP browser interface browser servers LAS XML configuration metadata UIS UI (java/script) HTTP Request (later SOAP) HTTP Response PS browser SOAP gif,netCDF,etc. BES BES BES UIS = User Interface Server PS = Product Server BES = Backend Server FDS = Ferret Data Server scripts &/or commands products Legacy OPeNDAP Legacy OPeNDAP addXML OPeNDAP Server data 2005 ESMF Workshop

  25. Architecture Components • User Interface Server (UIS) – keeps browser up-to-date with relevant interface widgets • Product Server (PS) – takes users requests and returns results • Backend Server (BES) – runs legacy application to fulfill request • Ferret Data Server (FDS) – General OPeNDAP server for LAS data 2005 ESMF Workshop

  26. The UI Server Architecture • User Interface Server • Installation process translates LAS configuration into User Interface Widgets need to present datasets, variables, geo-spatial and time extents in HTML • Allows dataset collection to be easily organized into any hierarchies the installer chooses • Communicates requests to the Product Server and returns responses (images, files, data tables, etc.) to the Web Browser • Many other customization features 2005 ESMF Workshop

  27. The Product Server • Product Server • Has knowledge of the LAS configuration • Makes requests from one or more Back-end Servers • Fully implemented Web-services interface will be available to client writers • Can interact with a series of Backend-servers to fulfill a single request from the user client • Back-end server • Can produce product itself or • Interact with a single legacy application (like CDAT or Ferret) to produce product • Requests are communicated via Request and Response XML that is exchanged via SOAP. 2005 ESMF Workshop

  28. FDS (Ferret Data Server) • FDS is an OPeNDAP server • Automatically installed with LAS • Therefore every gridded LAS dataset is served via OPeNDAP • Has extended analysis capabilities beyond basic OPeNDAP servers 2005 ESMF Workshop

More Related