1 / 37

BoM / CAWCR. Text Generation in the Next-Gen Forecast System (GFE)

BoM / CAWCR. Text Generation in the Next-Gen Forecast System (GFE). J Bally & T Leeuwenburg. Background & Drivers.... Next-Gen Forecast System. Better use of NWP models Systematic forecast process Temporal and spatial detail Can verify everything Efficiency gains

rance
Download Presentation

BoM / CAWCR. Text Generation in the Next-Gen Forecast System (GFE)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BoM / CAWCR. Text Generation in the Next-Gen Forecast System (GFE) J Bally & T Leeuwenburg

  2. Background & Drivers.... Next-Gen Forecast System • Better use of NWP models • Systematic forecast process • Temporal and spatial detail • Can verify everything • Efficiency gains • Many new services: grids, graphics and text all from the same weather database

  3. Nowcast: TIFS (objects) On-the-fly, shallow, slot filling

  4. Text Generation… introduction Most sophisticated meteorological text generation system ??? Large jump from “slot filling” systems (TIFS, TC, Scribe etc) Text as a network of nodes Goal directed multi-pass processing 64,000 lines of python - > 15 p-yr development

  5. Text Generation : example goals Try for <= three weather sub-phrases (2 for wind etc.) Describe the weather trends, rather than a sequence Describe changes in weather only if the impact differs substantially Try for elegant sentence structure; split out unusual weather types if they are not part of the trend Must-goals (guarantees) vs should-goals ……….etc etc

  6. Text Generation…multi-pass processing

  7. Text Generation…multi-pass processing

  8. Text Generation…multi-pass processing

  9. Text Generation.... overview Information representation Data Gathering Information Processing and Document Planning Mapping to Words ( Surface Realisation ) Post Processing

  10. Information Representation: Scalars, Vectors, Weather…… Sky PoP Weather Temp / Wind

  11. Information Representation: Hazards Hazards

  12. Text Generation.... Information representation Data Gathering Information Processing and Document Planning Mapping to Words ( Surface Realisation ) Post Processing

  13. Data Gathering.... Grid sampling • What about weather and hazards? • How to summarise a bit of patchy rain, isolated severe thunderstorms and raised dust? • Lets concentrate on the weather ..........

  14. Data Gathering.... Grid sampling- eg 3 hr time slices NoWx Sct SH - • Wide SH m Sct TS n • Patchy RA m } Isolated Showers } Isolated Thunderstorms Reported coverage = Σ (internal coverage * grid point count) total points

  15. Data Gathering.... Grid sampling NoWx Sct SH - • Wide SH m Sct TS n • Patchy RA m Reported coverage = Σ (internal coverage * grid point count) total points Reported Intensity = Σ (intensity contribution* grid point count) total affected grid points Similar calculation to collapse similar weather types…Sh/Dz/Ra

  16. Data Gathering.... Grid sampling NoWx Sct SH - • Wide SH m Sct TS n • Patchy RA m Filtering the Weather List

  17. Text Generation.... Information representation Data Gathering Information Processing and Document Planning Mapping to Words ( Surface Realisation ) Post Processing

  18. Information Processing.... Embedded Local Effect > Winds: Easterly 10 to 20 knots decreasing to 10 to 15 knots around midday then increasing to 15 to 20 knots during the afternoon, locally up to 30 knots in the east. Seas: Below 0.5 metres increasing to 0.5 to 1 metres by early evening, locally up to 1.5 metres in the east. Forecast-Split Local Effect > In the east: Winds: Easterly 10 to 20 knots increasing to 20 to 30 knots during the afternoon. Seas: 0.5 to 1 metres, increasing up to 1.5 metres by early evening. Elsewhere: Easterly 10 to 20 knots decreasing to 10 to 15 knots around midday then increasing to 15 to 20 knots during the afternoon. Seas: Below 0.5 metres increasing to 0.5 to 1 metres by early evening.

  19. Information Processing.... Check for Local Effects …. Scalar Metrics

  20. Information Processing.... Check for Local Effects

  21. Text Generation…multi-pass processing

  22. Information Processing.... Pre-Process Weather...... Arrange statistics in time order; Combine where appropriate, maintaining ranges; Separate co-reportable types Subphrases before preProcessWx Subphrases after preProcessWx

  23. Information Processing.... Simplify Weather...... Collapse Ranges Subphrases before preProcessWx Subphrases after preProcessWx

  24. Information Processing.... Merge Weather …. telling little white lies After mergeOverlap: before mergeGap: Subphrases after mergeGap:

  25. Recall …multi-pass processing

  26. Information Processing.... Have we tried every processing step enough? Have we achieved our goals for level of detail? CanAdjust Detail by….. Looking for more local effects?.. Split forecast? More aggressive sub-phrase combining Coarser sampling strategy Start again !

  27. Text Generation.... Information representation Data Gathering Information Processing and Document Planning Mapping to Words ( Surface Realisation ) Post Processing

  28. Mapping to words.... Process Trends…. Recognise and Summarise trends Subphrases before ProcessTrends Subphrasesafter ProcessTrends

  29. Mapping to words.... Connectors Increasing / Decreasing Becoming / Tending Developing / Clearing • Winds W toNW’y at 15 to 25 knots tending W to SW’ly then increasing to 30 knots. • Isolated showers developing during the morning then increasing to heavy widespread rain…..

  30. Mapping to words.... Time reporting Transition (change verbs) Over-time (nouns) Mixed (trend verbs) • Winds W to NW’y at 15 to 25 knots tending W to SW’ly around noon then increasing to 30 knots. • MorningFog. Isolated showers developingduring the afternoon then increasing to widespread rain…

  31. Text Generation.... Information representation Data Gathering Information Processing and Document Planning Mapping to Words ( Surface Realisation ) Post Processing

  32. Post Processing.... Post-Process Phrases - string replacements to cover limitations - “band-aid”… eg Early frost. Early fog. >> Early frost and Fog. Remove repeated words eg W to NW’y winds becoming NW’ly

  33. Example District Forecast... inc local effects

  34. Products all forecasts are in XML ...

  35. QC.. with some help from our testing infrastructure ...

  36. Change Management .... Importance of specifications Agreed? policies Big change in the role of forecasters Forecaster edits for style and/or substance Change management

  37. The EndText Generation in the GFE

More Related