DWD GISC Node Current Status

1. DWD GISC NodeCurrent Status

2. Agenda • Welcome • Current status: Overview based on functional blocks • High Level Architecture • Web Interface • DAR Component • Catalogue systems, Metadata and identified issues • GISC-to-GISC communication • Experience with Blog solution and AFD • Schema for GISC backup scenario • Data Components • MSS, AFD and subscription management • Live Demo • Challenges during the Implementation • Q&A

3. High level architecture: Functional blocks Logging, Monitoring and Reporting Authorisation/Authentication services SubscriptionManagement Gray List Container WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources Input Management Receiver External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer

4. Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor HLA: Web Interface (User/Admin/Customer) Logging, Monitoring and Reporting Authorisation/Authentication services SubscriptionManagement Gray List Container WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources Input Management Receiver External DCPC DAR-Generator/Importer

5. HLA: Web Interface (User/Admin/Customer) • WebWerdis • Redesign of WebWerdis • Navigation structure / search functions • Internationalization • Integrate logging framework (necessary for monitoring) • Code clean-up • Integration with Geonetwork • Geonetwork • Customization of GUI

6. HLA: Web Interface (User/Admin/Customer) • Search by URL – ISO 23950 • Timo (WMO Secretariat) did a SRU implementation in Geonetwork • Solution is running in DWD DMZ • 3 day implementation workshop with Timo about SRU and jzkit at DWD • Direct integration into DWD GISC node

7. SubscriptionManagement Gray List Container • Datapool(s) • controller/manager • access adapter • data sources Receiver • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer High level architecture: Metadata Management Logging, Monitoring and Reporting Authorisation/Authentication services WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller Input Management External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic

8. High level architecture: Metadata Management • DAR Catalogue • Did extensively Performance Tests (Geonetwork 2.2, terraCatalog 2.3) • DWD formulated test cases • Contracted external partner for tests • Functional Tests • DAR Generator / Importer • Developed DAR metadata generator (template driven architecture) • External review of DAR metadata • Developed DAR importer • Issues we identified during our work • Tested DAR catalogues fulfilled most, but not all INSPIRE and WMO requirements • DAR Metadata • Paper with Request for Clarification (JMA, CMA and DWD) • Schematron rules for validating WMO profile would be very helpful

9. High level architecture: Metadata Management • Metadata Harvesting • CMA and DWD did several functional tests with OAI-PMH in 09/09 • Different implementations of OAI-PMH could not harvest each other • Deleted metadata records: different implementations • Standard vanilla implementations modify harvested metadata • Ongoing tests between CMA and DWD • Started January 2010 • Initial tests showed 1 record ~ 1 second  space for improvement • Backup strategy • Exchange of metadata files as products

10. High level architecture: Metadata Management

11. High level architecture: G2G communication Logging, Monitoring and Reporting Authorisation/Authentication services SubscriptionManagement Gray List Container WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources Input Management Receiver External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer

12. HLA: GISC-to-GISC communication • Collaboration CMA and DWD • Memorandum of Understanding signed end of last year • Two Workshops held • GISC-to-GISC communication tests with CMA • CMA and DWD installed Blog solution and AFD solution • Data sets with appropriated time profile (issue warning during model transfer) • Blog solution • CMA and DWD did several improvements on publishing and polling components • Usage of tar minimize latency of transfer • Necessary to include size of “metadata” (AtomFeed) into calculation of transfer volume • Identified several further improvements like prioritization, AtomFeed handling, logging • JMA has provided a new improved solution in 10/2009 • AFD solution • Average transport time between 4-5 sec during 24hour continuously test, warnings between 3-4 sec • In case of unsecured connection sometimes corrupted data occur

13. HLA: GISC-to-GISC communication backup • Technical specification requires a bilateral agreement from 2 GISC for a backup scenario • CMA and DWD prepared proposal for priorities in DAR metadata • First common draft of dissemination metadata (XML schema) • Need to prove in practice

14. SubscriptionManagement • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources High level architecture: Delivery components Logging, Monitoring and Reporting Authorisation/Authentication services Gray List Container WMO GISC-GISC Interface Input Management Receiver External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer

15. High level architecture: Delivery components • Message Switching System • REST Web-Service to manage and control Moving Weather • First proposal for generic Interface • Integration into WebWerdis (still under work) • Subscription Management • Controller: time and event triggered management (still under work) • Customization according to GISC-to-GISC backup necessary

16. Live Demonstration

17. TC 5 - Maintenance of Users, Roles, Authorization and Authentication • Search metadata • Access data • Show difference between authorized and unauthorized User • User and Customer management • Subscription • View historic transactions

18. Metadata Management – TC 1 • Show metadata editor • Fully integration open because of separation between Intranet and DMZ • Authorization required to add/modify/delete metadata

19. Metadata Management – TC 2, TC 6 • A WIS Centre provides new metadata records • Show that GISC Geonetwork doesn’t contains new metadata records • Show that GISC_PRODUCTS is empty • Copy new metadata records to remote provider, reindex • GISC harvests new metadata records • Update of GISC Geonetwork and jOAI Provider • Show update of Geonetwork with SRU query • Show update of GISC_PRODUCTS

20. Challenges during the Implementation • Metadata generation • See paper “Questions on Metadata” from CMA, JMA and DWD from 11.2009 • WIS Specification is open – what is good!

21. Q&A