Quo Vadit Visual Weather?

1. Quo Vadit Visual Weather? By JozefMatula

2. What is IBL Visual Weather? • Meteorological Workstation SW providing: • Met. data processing and visualisation • Interactive forecasting tools • Forecast production and workflow management • Batch production • Extensibility with Python API • Web services (WMS, WCS, WFS, JMBL) • Everything in one box or Client-Server • Highly configurable • 1.2M+ lines of C++, 120k+ lines of Python, 40k lines of XML, 120k lines of Fortran, 220k lines of configuration…

3. Developers have worked really hard during… past YEAR of VW development

4. 3, 2, 1, start! • 3 maintained development branches: • 3.1.x stable • 3.2.y stabilising • 3.3.z development • Most significant development areas: • Forecast production and workflow • Grid “dataset” concept • Field Diagnostics and Modification (UK MetMorph) • Batch production improvements • Tropical cyclone stuff • Web Services

5. The purpose of life is… FORECAST production

6. Forecast Production - Features • We refactored our editor of features (mainly for SIGWX requirements) • Every feature is timestamped and geolocated • Stored and shared in the “Feature Database” • Usability enhancements: • Single click drawing • Curve optimalisation(to reduce noise from tablet) • Cooperation withunderlying layers • Work on WFS-T access in progress

7. Example of a Feature Product With courtesy of 21 OWS USAFE, Sembach

8. Forecast Production – Text forecasts • XML Message Editor Templates (declarative language) • User input validation • Preview before issue • Internal production and delivery chain • Different outputs from one form • Single click for whole production and delivery • Revision management • Corrections, amendments and cancelations • Access to history of issued forms • Initialisation of form contents from • VW Mathematical Kernel (Observation, NWP data) • Plugins in Python

9. Forecast Production - Message Editor 2 PDF AFTN/GTS

10. Message Editor – Template Language • XML description language, consisting of: • “Model” – hierarchical data structure definition • “Layout” – mapping of model to GUI

11. Message Editor – Building Blocks Instance of data model can be transformed in multiple ways and delivered to multiple places Delivery Output Filters Text Bulletin Send to AFTN Form ODT w. all reports Print 2x copies Aerodrome Warning PDF for Station 1 FTP to Web PDF for Station 2 Copy to Archive PDF for Station 3 Production Pipeline

12. Message Editor – Embedded Custom UIs

13. Forecast Production – Workflow Management • Personal Task Manager provides a role based ToDo list interconnected with production applications Access to history of work Colour of task can be used to indicate type or priority of the task Hyperlink State of work Workflow actions

14. Well I am the recent forecast who is more? The “dataset” concept

15. Dataset – What for? • If you modify fields, you might want to see the previous revisions (On Screen Analysis, Field Modification) • EPS models require completely new dimension(s) in “grid space navigation” (perturbations, statistics fields, probability fields) Forecast Level Run base time Dataset

16. Datasets – EPS Example Dataset is also a global map variable

17. Tell me that this is not a magic… CONSISTENT FIELD Modification (MetMORPH)

18. Field Modification • We redesigned original simple field modification module into generalised field modification framework • 2 available implementations: • Old style single-field modification • UK Met Office MetMorph (successor of OSFM) • Output is stored in GRIB and available to visualisation or batch production • Comparing to UKMO Horace 4 MetMorph: • Facelifted and integrated • Removed some original limitations • Access to history of changes

19. MetMorph – Main Features • Global and mesoscale model merging • Dynamical field change • Editing of wind • Editing of precipitation (enhancing, reduction, wind advection) • Merging and advecting precipitation from radar • Orographic modulation • Field spatial and time smoothing and interpolation • Variation of changes in time • Shape and change-vector morphing • Variation of change-strenght in time

20. Met Morph – Dynamical MSLP change

21. MetMorph – Time Linking

22. Web server cannot imagine his life without… BATCH PRODUCTION IMPROVEMENTS

23. Batch production improvements • Parallel execution of batch jobs • Ability to have “user” and “role based” batch jobs and limit triggers. • Improved triggers for: • Detection of particular NWP parameter & forecast • Area/station list based monitoring of observations • Detecting completion of a Feature Product • New jobs: • Field Diagnostics – precalculation of grid fields algorithms • Field Cutting • BatchOSA, Batch Message Editor production

24. Also an umbrella is one of… Tropical cyclone forecasting tools

25. Tropical Cyclone Forecasting Tools • Just an experimental internal development – both implemented in Python as a kind of tutorial sandbox example: • TC Storm tracking visualisation (parsing the free text messages from GTS) • Dvorak helper construction

26. Tropical Cyclone Forecasting Tools - Examples

27. One day if I want to cook an egg I’ll make for it use of… Web Services

28. Present Web Services in Visual Weather • Multi threaded with control of maximum load • Implemented as yet another VW service • WMS: • Exposes ANY product from visualisation • Configurable dimensions and layer styles • WCS: • Exposes any grid field (or derived field) from VW database, including EPS fields. • WFS: • Access to TAF/METAR reports • We see a lot of potential here (everything can be seen as a feature!) • JMBL – JMOBS and JMGRID

29. OGC WS implementation dilemmas • The challenge for building web applications is to find the proper border between server and client, and to choose the proper protocol • Also there are some technical problems: Compression vs. transparency Rotated WAFC projections Depending dimensions MIME types Non-geospatial projections 2D time Date line Dimension naming Dynamic capabilities CRSs vs. GRIB/GeoTIFF Lack of “good” clients Elevation units Absence of GML schemas

30. Visual Weather WMS “specialities” • PROJ4: and IBL: CRS namespace CRS=PROJ4:+proj=latlong +datum=NAD83 CRS=IBL:TEPHIGRAM • Cross-sections • Do you know CRS:1? • How to pass the “place” information: • Horizontal route: DIM_PLACE=LZIB EPSG:4326[48;19] EHAM • Time crosssection DIM_PLACE=EPSG:4326[48;19] • JPEG2000 (good compression with alpha transparency) – but lack of support in browsers/Flex – the same with GeoTIFF

31. Visual Weather W*S dimensions • For WMS naming and dimensions are configurable: • No dimensions at all (latest data use case) • Single TIME dimension (both for observed or forecasted data from the recent model run) • DIM_RUN+TIME dimension – doesn’t look to be a good way (non-orthogonal dimension space) • DIM_RUN+DIM_FORECAST • Optional ELEVATION dimension with configurable ways how to format the values (850 vs. 850hPa).

32. Visual Weather W*S dimensions (2) • For WCS we have only static dimensions: • DIM_RUN, DIM_FORECAST, TIME, (+ EPS DIM_MEMBER) – time dimensions overlap • For WFS – noone cares about dimensions anymore – it’s like single table SQL… • WCS and especially WFS are vulnerable to DoS attacks!

33. RIA powered by pure VW WMS

34. Well and something more… 1600 other improvements, Changes and Fixes 

35. Thank youfor your attention!Questions?

36. Thanks to KNMI and particularly Kees and Helen for the hospitality!