530 likes | 729 Views
Jaime Clark and Bianca Gibson Brianna Laugher Tennessee Leeuwenburg December 2012. Forecast text editing rates and analysis. www.cawcr.gov.au. Graphical Forecast Editor. GFE Text formatters. GFE Text formatters - editing. GFE Forecasting. Guidance. Grid Editing & Smart Tools.
E N D
Jaime Clark and Bianca Gibson Brianna Laugher Tennessee Leeuwenburg December 2012 Forecast text editing rates and analysis www.cawcr.gov.au
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Graphical Forecast Editor
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Text formatters
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Text formatters - editing
GFE Forecasting Guidance GridEditing & Smart Tools Text Formatting Forecast Production The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology
GFE Forecasting Text Formatting Forecast Production Grids original editedtext gfemetrics The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics http://tardis.bom.gov.au/gfemetrics Web front-end to “FDB” database created/maintained by ITB Initially developed by Brianna, further developed by Anthony Briggs STATS/SUMMARY Can view graphs/tables of total edits: by time, by state, by formatter VIEW Can view any edits made to any GFE forecasts Can query edits by state, date, formatter, period, issuance, locale Can search for edited/unedited/either containing particular words TOOLS Can view graphs/tables of most used tools: by time, by state
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Metric? We initially hoped that improvements to the formatters would be reflected in lower edit rates... this appears not to be the case... but it's hard to tell...
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Dimensions of edits <—————— REPRESENTATIVENESS ——————> Is the generated text consistent with grid contents? “Wrong” results (genuine error) vs stylistic changes (subjective) “Possible shower” -> “Mostly dry” <—————— POLICY ——————> Is the edited text consistent with Services policy? Policy may need resolving with Services “Fine”, “moderating”, “freshening” <—————— MISSING DATA ——————> Is the info required to produce the edited text present in the GFE? Formatter missing rules - “Humid”, “Hot”, sea breeze GFE missing grids – climatology, uncertainty! Other info in forecaster brain (local expertise) Should fixASAP Can't fix(for everyone) Won't fix Can potentially fix
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Limitations Formatters are mainly just a group of phrases – Sky, Wx, Wind, Temp, Seas, Swell. Effort is directed at improving phrases – but there's no way to check edit rates of phrases Hard to get a sense of any general trend when a query returns hundreds of results Little/no time to devote to exploring regional differences
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Monash Talented Students program Science-Industry Student Research Placement Program “short-term, research-based industry placements”Rationale (i) facilitating industry experience for upper undergraduate students; (ii) building collaborations in work integrated learning between Monash and industry partners; (iii) furthering science and its broader communication.
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology About Bianca Second year in undergraduate Computer Science
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Analysing edits by formatter Per forecast period Per state Percentages
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Marine edits by forecast period - percentage
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Marine edits by forecast period - totals
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Precis edits by forecast period - percentages
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Town, metro and district edits by forecast period - percentage
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology NSW edits by formatter - percentage
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology NSW edits by formatter - totals
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology SA edits by formatter - percentage
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology TAS edits by formatter - percentage
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology VIC edits by formatter - percentage
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology WA edits by formatter - percentage
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Two-Town comparisons Hypothesis: Bigger towns (capital cities) get more attention than smaller towns Does “the gap” represent editing that would be done if forecasters had infinite time? Or does it represent stylistic editing that could be done without?
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Two-Town comparisons - VIC
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Two-Town comparisons - TAS
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparisons Hypothesis: initial edits very low, increase as forecasters learn to use the GFE and have more time for text editing “Month 0” is the month the GFE went live in that region VIC: 5/2009 NSW: 9/2010 TAS: 6/2011 SA: 11/2011 WA: 10/2012 Each state has a different version of the GFE and different weather conditions The dot and percentage on the far right represents edits in 11/2012
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison – all formatters (incl. warning)
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison - Precis
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison - Metropolitan
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison - marine
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology About Jaime 4th year in double undergraduate degree: Science Major - Pure Mathematics Minor - Developmental Psychology Minor – Computational Mathematics Computer Science Major - Intelligent Systems Major – Software Engineering 5 year course
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Edits per formatter by phrase type edited How do we do this? Formatter Unedited forecast Edited forecast Break into sentences Examining key known words within the sentence to determine type Identify the sentence type Unedited categorized sentences Edited categorized sentences Compare categories: Identify if a change is present
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Town 01-NOV-2012 --- 30-NOV-2012 :: TOWN_FORECAST
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - District 01-NOV-2012 --- 30-NOV-2012 :: DISTRICT_FORECAST
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Metropolitan 01-NOV-2012 --- 30-NOV-2012 :: METROPOLITAN_FORECAST
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Coastal 01-NOV-2012 --- 30-NOV-2012 :: COASTAL_FORECAST
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited – Local Waters 01-NOV-2012 --- 30-NOV-2012 :: LOCAL_WATERS_FORECAST
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Marine 01-NOV-2012 --- 30-NOV-2012 :: MARINE_FORECAST
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Precis: Investigating additions/removals How do we do this? Formatter Unedited forecast Edited forecast Identify if the given key is present Unedited keys identified Edited keys identified Compare keys: Identify if a key has been added/removed
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Ratio to un-edited Text
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Morning – strongly removed 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Afternoon – strongly added 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Late – strongly added 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Developing – strongly removed 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Clearing – strongly removed 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT
The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Towns - Haze/Humid? 01-FEB-2012 --- 29-FEB-2012 :: PRECIS_TEXT The forecaster trick to get humid into the forecast: • Add haze into the grids. • Generate forecasts. • Remove haze from the grids. • Complete a search and replace all: haze to humid.