1 / 20

ESG Product Services Overview of components Issues in need of discussion

ESG Product Services Overview of components Issues in need of discussion. Steve Hankin, NOAA/PMEL Roland Schweitzer, Weathertop Consulting. Questions for consideration. Gateway Node versus Data Node components trade-offs: performance, complexity & system loading

zaynah
Download Presentation

ESG Product Services Overview of components Issues in need of discussion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ESG Product ServicesOverview of componentsIssues in need of discussion Steve Hankin, NOAA/PMELRoland Schweitzer, Weathertop Consulting ESG All-Hands meeting

  2. Questions for consideration • Gateway Node versus Data Node components • trade-offs: performance, complexity & system loading • AuthN & AuthZ implications of legacy back ends • How do we do "batch" products in ESG? • How to handle curvilinear data? • Ajax connections to the UI -- AuthN & AuthZ issues • What ESG component should generate CF file subsets? • Public access to OPeNDAP. To server-side calculations? • NCL back end and updates to CDAT ESG All-Hands meeting

  3. URL command file XMLmetadata binarydata .dds?DIF .dods?DIF F-TDS URL: “SST from COADS minus TEMP from Levitus (regridded)” http://host.noaa.gov/thredds/data/coads_expr_{levitus.nc}{DIF=SST-TEMP[g=SST]} Data-Source TDS(or BES?) 1. write IOSP Let DIF =SST-TEMP[g=SST] 2. run read Ferret write F-TDS Server-sidetransformations ESG All-Hands meeting

  4. AJAXAPI RESTAPI Backend Service 2 Backend Service 3 Backend … BackendService RDBMS e.g. WMS … ‘Legacy’ application(e.g. graphics …NCL, CDAT, Ferret, NCO, …) F-TDS+ LocalData Server-sidedata reduc’n,regrid, diff’ing product request XML (REST) UI Metadata (AJAX) back endrequest (SOAP) OPeNDAPDAP (REST) LAS a “workflow engine”: reads data/makes products Product Server(Java) Metadata(XML) user interface Remote Data

  5. AJAXAPI RESTAPI Backend Service 2 Backend … Backend Service 3 BackendService RDBMS e.g. WMS … ‘Legacy’ application(e.g. graphics …NCL, CDAT, Ferret, NCO, …) F-TDS+ LocalData Server-sidedata reduc’n,regrid, diff’ing product request XML (REST) UI Metadata (AJAX-REST) back endrequest (SOAP) OPeNDAPDAP (REST) LAS Product Server(Java) Metadata(XML) user interface Remote Data If no xforms, regridding or differencing arte needed ESG All-Hands meeting

  6. publication Backend legacy app. product request(REST) UI Metadata (AJAX) OPeNDAPDAP (REST) back endrequest (SOAP) Level 5 • Differencing occurs at the Data node • Distinct graphics by institution • Server-side data reduction Gateway Node Product Server Product Server Data Node Portal UI F-TDS+ Data Node 2 Data Nodes talk to other Data Nodes when differencing ESG All-Hands meeting

  7. publication Backend Backend legacy app. legacy app. product request(REST) UI Metadata (AJAX) OPeNDAPDAP (REST) back endrequest (SOAP) Level 4 • Differencing occurs at the Gateway node • Distinct graphics by institution • Server-side data reduction Gateway Node Product Server Product Server Data Node Portal UI F-TDS+ F-TDS Data Node 2 Data Nodes don’t talk to other Data Nodes ESG All-Hands meeting

  8. publication Backend Backend legacy app. legacy app. product request(REST) UI Metadata (AJAX) OPeNDAPDAP (REST) back endrequest (SOAP) Level 3 • Differencing occurs at the Gateway node • Distinct graphics by institution • Server-side data reduction Gateway Node Product Server Data Node Portal UI F-TDS+ F-TDS Data Node 2 Data Nodes doesn’t have full LAS ESG All-Hands meeting

  9. publication Backend legacy app. product request(REST) UI Metadata (AJAX) OPeNDAPDAP (REST) back endrequest (SOAP) Level 2 • Differencing occurs at the Gateway node • Uniform graphics for all inst’ns (or complx Gateway) • Server-side data reduction Gateway Node Product Server Data Node Portal UI F-TDS+ F-TDS Data Node 2 ESG All-Hands meeting

  10. publication Backend legacy app. product request(REST) UI Metadata (AJAX) OPeNDAPDAP (REST) back endrequest (SOAP) Level 1 • Differencing occurs at the Gateway node • Uniform graphics for all inst’ns (or complx Gateway) • NO Server-side data reduction Gateway Node Product Server Data Node Portal UI OPeN-DAP F-TDS xform, regrid, diff. aggregation, only Data Node 2 Absolute minimum data node ESG All-Hands meeting

  11. ESG Product ServicesTopics for discussion • Gateway Node versus Data Node components • trade-offs: performance, complexity & system loading • AuthN & AuthZ implications of legacy back ends • How do we do "batch" products in ESG? • How to handle curvilinear data? • Ajax connections to the UI -- AuthN & AuthZ issues • What ESG component should generate CF file subsets? • Public access to OPeNDAP. To server-side calculations? • NCL back end and updates to CDAT ESG All-Hands meeting

  12. Product Server Data Node Level 5 publication publication F-TDS+ Backend Backend legacy app. legacy app. differencing Data NodeLevel 2 product request(REST) UI Metadata (AJAX) F-TDS+ OPeNDAPDAP (REST) back endrequest (SOAP) • Gateway Node versus Data Node components Recommendation: Level 2 – simple Level 5 – high power Gateway Node Product Server Portal UI F-TDS+ ESG All-Hands meeting

  13. publication publication Product Server Data Node Level 5 F-TDS Backend Backend legacy app. legacy app. differencing product request(REST) UI Metadata (AJAX) Data NodeLevel 2 OPeNDAPDAP (REST) back endrequest (SOAP) F-TDS • AuthN & AuthZ implications of legacy back ends Relink withESG-aware ‘c’ client libraries? Gateway Node Product Server Portal UI F-TDS ESG All-Hands meeting

  14. publication publication Product Server Data Node Level 5 F-TDS Backend Backend legacy app. legacy app. differencing product request(REST) UI Metadata (AJAX) Data NodeLevel 2 OPeNDAPDAP (REST) back endrequest (SOAP) F-TDS • How do we do "batch" products in ESG? How to retain Auth info when picking up delayed response? Gateway Node Product Server Portal UI F-TDS What if F-TDS response will be delayed? ESG All-Hands meeting

  15. publication publication Product Server Data Node Level 5 F-TDS Backend Backend legacy app. legacy app. differencing product request(REST) UI Metadata (AJAX) Data NodeLevel 2 OPeNDAPDAP (REST) back endrequest (SOAP) F-TDS • How to handle curvilinear data? Gateway Node Product Server Portal UI F-TDS Diagnostic vars often must be computed on the native grids? Need to get a list of all likely diagnostic vars for AR5 in advance ESG All-Hands meeting

  16. publication publication Product Server Data Node Level 5 F-TDS Backend Backend legacy app. legacy app. differencing product request(REST) UI Metadata (AJAX) Data NodeLevel 2 OPeNDAPDAP (REST) back endrequest (SOAP) F-TDS • Ajax connections to the UI -- AuthN & AuthZ issues? Anything to be worried about? Gateway Node Product Server Portal UI F-TDS ESG All-Hands meeting

  17. publication publication Product Server Data Node Level 5 F-TDS Backend Backend legacy app. legacy app. differencing product request(REST) UI Metadata (AJAX) Data NodeLevel 2 OPeNDAPDAP (REST) back endrequest (SOAP) F-TDS • What ESG component should generate CF file subsets?(so-called ‘fileout’) As another product’? A Gateway Node capability ? Gateway Node Product Server Portal UI F-TDS Another REST protocol on the OPeNDAP server? What about ‘batch’? ESG All-Hands meeting

  18. publication publication Product Server Data Node Level 5 F-TDS Backend Backend legacy app. legacy app. differencing product request(REST) UI Metadata (AJAX) Data NodeLevel 2 OPeNDAPDAP (REST) back endrequest (SOAP) F-TDS • Public access to OPeNDAP. Server-side calculations, too?(a policy question) Gateway Node Product Server Portal UI F-TDS Denial ofservice risk? ESG All-Hands meeting

  19. Product Server Data Node Level 5 publication publication F-TDS+ Backend Backend legacy app. legacy app. differencing Data NodeLevel 2 product request(REST) UI Metadata (AJAX) F-TDS+ OPeNDAPDAP (REST) back endrequest (SOAP) • NCL back end and updates to CDAT(just a reminder) NCL and CDAT with SOAP wrappers? Gateway Node Product Server Portal UI F-TDS+ ESG All-Hands meeting

  20. Whew!(the end) ESG All-Hands meeting

More Related