1 / 19

PP INSPECT report

PP INSPECT report.

lweidner
Download Presentation

PP INSPECT report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PP INSPECT report Dmitry Alferov (1), Elena Astakhova (1), Petra Baumann (4), Dimitra Boukouvala (2), Anastasia Bundel (1), Ulrich Damrath (3), Pierre Eckert (4), Flora Gofa (2), Alexander Kirsanov (1), Xavier Lapillonne (4), Joanna Linkowska (5), Chiara Marsigli (6), Andrea Montani (6), Anatoly Muraviev (1), Elena Oberto (7), Maria Stefania Tesini (6), Naima Vela (7), Andrzej Wyszogrodzki (5), and Mikhail Zaichenko (1), André Walser (4) (1) RHM (a.bundel@gmail.com), (2) HNMS, (3) DWD, (4) MCH, (5) IMGW-PIB, (6) ARPA-SIMC, (7) ARPA-PT COSMO GM, 11-15 September 2017, Jerusalem, Israel

  2. Spatial methods Displacement methods Filtering methods Many√ • Features-based • Contiguous Rain Area (CRA) (Ebert and McBride, 2000) • Method for Object-based Diagnostic Evaluation (MODE) (Davis et al., 2006) •  Structure, Amplitude, Location (SAL) (Wernli et al., 2008) • Field deformation • Neighborhood • (Ebert, 2008) • Scale Decomposition • DIST method √ √ √ √ √ √ √ √ √ Now almost all the categories of spatial methods are applied by INSPECT participants. Flora Gofa: “It gave us a chance to learn methods that before we just knew by name”. Thus, the main benefit of INSPECT “that the wide range of spatial verification methods available will become commonly used within the COSMO community” is achieved.

  3. Status highlights • The PP INSPECT is extendeduntil the end of 2017 due to delays in some tasks, with 0.3 FTE shifted and 0.2 FTE added • Research tasks are mostly completed except for some tasks involving ensembles. • The reports describing the properties of each spatial method are written by participants. The report summarizing them is under preparation.

  4. Reruns for MesoVICT test cases • MCH: COSMO-1 reruns for ALL MesoVICT cases are done and available at WG5 repository (Petra Baumann) • ECMWF-IFS reruns (51 member) for cases 1 and 2 (8 initial dates) (Andrea Montani), • COSMO-E reruns (21 member) for cases 1 and 2 (8 initial dates) (André Walser), • COSMO-Ru2-EPS (51 member) for case 1 (1 initial date) and case 2 (is running now) (Dmitry Alferov) All reruns are interpolated to VERA grid by Manfred Dorninger (Austria) -> easy to use and compare!

  5. Tasks involving development of routines for neighborhood, CRA, SAL, and MODE • For the most part, the software is based on free R SpatialVx package (developed by E. Gilleland). For SAL (D.Boucoucala) and Neighborhood (J.Linkowska) comparisons are made with alternative packages -> bug fixing of SpatialVx • VAST development by N. Vela (ARPA-PT):inclusion of time dimension and variables besides pecipitation, namely, cloud cover and wind speed

  6. Neighborhood applications.Analysis of long time series (DWD, MCH) Relax the requirement for an exact match by evaluating forecasts in the local neighborhood of the observations

  7. Uli Damrath’s method to aggregate the scores • Getting Equitable Threat Score (ETS) for upscaling and Fractions skill score (FSS) as monthly values from fuzzy verification • No averaging over daily values but calculation of scores from the contingency tables of the whole month • Calculation of running means of the results over one year • Presentation of mean values and mean averages

  8. Comparison of COSMO-EU to COSMO-DE – upscaling Equitable Threat Score (ETS) Precipitation amount W i n d o w s I z e Threshold 0.1 0.2 0.5 1 2 5 10 20 50 1.625 0.825 0.425 0.225 0.125 0.075 0.025 Mesh width of COSMO-EU

  9. Compact visualization of total precipitation FSS: to focus on the useful scale for a given lead-time and threshold (MCH) FSS: Fractions Skill Score P is the event fraction in the neighborhood.

  10. Intensity-scale: filtering method (F. Gofa, HNMS) Error image is expressed as the sum of components on different spatial scales by performing a 2-dimentional discrete Haar wavelet decomposition. The spatial scale refers to the spatial scale of the error and not that of the precipitation features as it happens in the neighborhood methods. MesoVICT case 1: 20070621-15, Intensity Scale Skill COSMO2 COSMO1 Small scales have skill close to zero, while large scales exhibit large skill. COSMO1 ISS graphs exhibit that errors due to displacements of small spatial scale features are more important compared to those of COSMO-2

  11. Case study2007.09.25.06, 6h precipitation, threshold>=5mm Object-based methods by IMGW-PIB • MODE, CRA, SAL VERA COSMO 2 unmatchedobject (false) Selected feature pairings based on total interest obs feature mod feature total interest 1 1 0.898

  12. SAL: Maria Stefania Tesini and Daniele D'Alessandro A single parameter to evaluate the structure, amplitude, or location error in forecast is not enough when the precipitation field complexity is too high Precipitation intensity is overestimated in Germany and underestimated in France, but the amplitude A is close to zero

  13. Object matching for EPS, MesoVICT case 1 (A.Bundel) Probability of each observed object is found and the ensemble skill can be estimated using the BSS, for example 2007062021 COSMO-E ensemble, first 6 of 21 members, precip threshold >0.5 mm/1h Probabilities of each of 5 observed objects: 1/21 20/21 10/21 19/21 14/21 Difficulty of such an approach: we have estimate only probabilities of objects CRA errors In spatial shift, precip volume, and fine-scale pattern can be

  14. SAL for EPS: ways to display large amount of ensemble data (Dimitra Boucouvala) 20/6 20/6

  15. SAL for EPS: introducing observation uncertainty (Dimitra Boucouvala) Objects comparison for probability of precipitation >= 2mm Probability threshold =1 LEPS Observations Preci >= 2 mm Preci >= 2 mm for all 16 members 3 h Precipitation 21/6 12 UTC S=1, A=0.38, L=0.3

  16. Andrea Montani: Sensitivity of COSMO-LEPS forecast skill to the verification network: application to MesoVICT cases

  17. New ensemble precipitation observation product by MeteoSwiss Available for the past data (e.g. for Mesovict cases) Available for Swiss + whole alpine domain for daily accumulation

  18. Main results • Several ways of compact visualization of long time series of neighborhood scores are proposed (DWD, MCH) • The object-based SAL (Structure-Amplitude-Location) method is easier to implement as it doesn’t require pair-wise matching of observed and forecast objects. However, it can give misleading results as it estimates characteristics in the whole domain, e.g. if precipitation intensity is overestimated in one place and underestimated in another, the amplitude value will be close to zero. • The object-based MODE and CRA methods provide more information as the matched pairs of observed and forecast objects are compared. However, it’s difficult to find a best universal matching function, in particular for high-resolution fields with objects of complex shape. • The DIST, SAL and CRA methods are applied to ensembles (ARPAE-SIMC, HNMS, RHM) • The experiments are started on introducing observation uncertainty into the spatial methods (ARPAE-SIMC, HNMS, MCH) The main benefit of INSPECT is achieved: the most wide-spread spatial methods enter into the everyday practice within the COSMO Community

  19. Thank you for your attention !

More Related