1 / 55

Evaluation of year 2004 monthly GlobAER aerosol products

This evaluation assesses the performance of GlobAER aerosol products in 2004, focusing on aerosol optical depth (AOD) and Angstrom parameter. The study compares the data to trusted references and existing datasets, examines bias and variability, and quantifies performance using a scoring system.

hassen
Download Presentation

Evaluation of year 2004 monthly GlobAER aerosol products

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Evaluationof year 2004 monthly GlobAERaerosol products Stefan Kinne

  2. the task • an evaluation of • GlobAER 2004 global maps • for aerosol optical depth (info on amount) • for Angstrom parameter (info on size) • by ASTR (dual view, global, at best 10 per month) • by MERIS (nadir view, global, at best 1 per day) • by SEVIRI (nadir view, regional, at best 30 per day) • by a merged (ATSR / MERIS / SEVIRI) composite

  3. the questions • how well do the data compare to trusted data references (e.g. AERONET) ? • how well do the data compare to existing data sets – even for same sensor data ? • can the performance be quantified ? • more specifically … • what are the scores of a new (outlier-resistant) method … examining • bias, spatial and temporal variability ?

  4. the investigated properties • aerosol optical depth (AOD) • extinction along a (vertical) direction due to scattering and absorption by aerosol • here for the entire atmosphere • here for the mid-visible (0.55mm wavelength) • Angstrom parameter (Ang) • spectral dependence of AOD in the visible spectrum • small dependence (Ang ~ 0) a aerosol > 1mm size • strong decrease (Ang > 1.2) a aerosol < 0.5mm size

  5. GlobAER 2004 maps GAa - ATSR GAs - SEVIRI GAm - MERIS GAx – merged other multi-ann. maps med – model median clim – med & aer(sun) sky – med & aer(sky) TO – TOMS other 2004 maps ATs - ATSR Swansea SEb- SEVIRI Bruxelles Mdb - MODIS deep blu MO - MODIS std coll.5 MI – MISR version 22 Ag – AVHRR, GACP Ap – AVHRR, Patmos Aer – AERONET 2004 monthly data-sets

  6. AOD map comparisons • AOD annual maps • all available data • AOD seasonal maps • ATSR GlobAER vs Swansey • SEVIRI GloabAER vs RUIB-Bruexelles • difference maps … to a remote sensing ‘best’ composite • all available data-sets • focus on the four GloabAER products

  7. AOD – 2004 annual maps

  8. ATSR / SEVIRI – seasonal AOD

  9. AOD diff to ‘composite’  underestimates overestimates 

  10. quick (annual) AOD check • ATSR • underestimates in dust regions • overestimate in biomass regions • SEVIRI • severe biomass overestimates • useful over-land estimates ? • MERIS • apparent land snow cover issue • merged • not the envisioned improvement  ‘-’ ’+’ 

  11. Angstrom map comparisons • Angstrom annual maps • all available data • Angstrom seasonal maps • ATSR GlobAER vs Swansey • SEVIRI GloabAER vs RUIB-Bruexelles • difference maps … to a climatology (model & AERONET) • all available data-sets • focus on the four GloabAER products

  12. Angstrom – 2004 annual maps

  13. ATSR / SEVIRI – seasonal Angstr.

  14. Angstrom diff to ‘climatology’ . .  underestimates overestimates 

  15. quick annual Angstrom check • ATSR • underestimates in tropics • overstimates in south. oceans • SEVIRI • underestimates over oceans • strong overestimates over land • MERIS • overestimates over land • merged • not the envisioned improvement  ‘-’ ’+’ 

  16. the SCORING challenge • quantify data performance by one number • develop a score such that contributing errors to be traceable back to • bias • spatial correlation • temporal correlation • spatial sub-scale (e.g. region) • temporal sub-scale (e.g. month, day) • make this score outlier resistant

  17. one number ! - 0.504

  18. info on overall bias - 0.504 sign of the bias

  19. | 1 | is perfect …. 0 is poor - 0.504 sign of the bias the closer to absolute 1.0 … the better

  20. product of sub-scores - 0.504 = 0.9 *- 0.7 * 0.8 temporal correlation sub-score bias sub- score spatial correlation sub-score the closer to absolute 1.0 … the better sign of the bias

  21. spatial stratification - 0.504 = 0.9 * -0.7 * 0.8 overall score time score bias score spatial score regional surface area weights spatial sub-scale scores TRANSCOM regions

  22. temporal stratification - 0.504 = 0.9 * -0.7 * 0.8 overall score time score bias score spatial score spatial sub-scale scores averaging in time instantaneous median data temporal sub-scale scores (e.g. month or days)

  23. sub-score definition • each sub-score S is defined • by an error eand • by an error weight w 0.9 * -0.7 * 0.8 S = 1 – w* e time score S bias score S spatial score S spatial sub-scale scores instantaneous median data temporal sub-scale scores (e.g. month or days)

  24. definition of errors e • S = 1 – w* e • all values for the errors e are rank - based for “time score” and “spatial score” rank correlation coefficients for data pairs are determined • e, correlation = (1- rank_correlation coeff.) /2 (correlated: e = 0, anti-correlated: e = 1) time score bias score spatial score

  25. definition of errors e • S = 1 – w* e for “bias score” all all data-pairs are placed in a single array and ranked by value then ranks are separated according to data origin, summed and (rank-sums are) compared • e, bias= (sum1 – sum2) / (sum1 + sum2) (strong neg.bias e = -1, strong pos.bias e= +1) an example (“how does the rank bias error work ?”) • set 1: 1 7 8 value: 98 7431 rank-sum 1: 11 • set 2: 3 4 9 rank: 1 2 3 4 5 6 rank-sum 2: 10 e = (1-2)/(1+2) = (11-10)/21 ~zero a no clear bias time score bias score spatial score

  26. definition of error weight w • S = 1 – w* e • wis a weight factor based on the inter-quartile range / median ratio • w = (75%pdf - 25%pdf) / 50%pdf … but not larger than 1.0 (w<1.0) simply put … if there is no variability an error does not matter time score bias score spatial score

  27. scoring summary • one single score … • … without sacrificing spatial and temporal detail ! • stratification into error contribution from • bias • spatial correlation • temporal correlation • robustness against outliers • still … just one of many possible approaches • now to some applications …

  28. questions • how did GlobAER products score? • overall ? • seasonality ? • spatial correlation ? • bias ? • in what regions ? • in what months ? • how did scores place to other retrievals … • with the same sensor (for the same year 2004) • with other sensors (for the same year 2004)

  29. selective evaluations for year 2004 data • ..atsr Swansey ATSR • ..gaat GlobAER ATSR 3/2009 (std) • ..gaa2 GlobAER ATSR 7/2009 (filtered) • ..gaa3 GlobAER ATSR 8/2009 (test) • ..misr MISR ver.22 • how does ‘gaa2’ score ? • score diff between ‘gaa2’ and ‘misr’ • score diff between ‘gaa2’ and ‘gaat’ • score diff between ‘gaa2’ and ‘atsr’

  30. performance vs AERONET * too few areas with scores

  31. side by side let us compare AOD details … • what is the sign of the bias ? • what is the bias strength (median diff) ? • what is the bias error ? • what is the spatial variability error ? • what is the seasonality error ? • what is the overall error • what are selected error differences among different retrieval products?

  32. AOD bias sign vs AERONET ATSR GlobAER ATSR Swansey ATSR GlobAER filtered MISR vers.22 underestimate overestimate

  33. AOD bias error vs AERONET ATSR GlobAER ATSR Swansey MODIS coll.5 MISR vers.22

  34. AOD bias strength vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22

  35. AOD bias error vs AERONET ATSR GlobAER ATSR Swansey ATSR GlobAER filtered MISR vers.22

  36. AOD spatial var. Vs. AERONET ATSR GlobAER ATSR Swansey ATSR GlobAER, filtered MISR vers.22

  37. AOD season error vs AERONET ATSR Swansey ATSR GlobAER MISR vers.22

  38. AOD total error vs AERONET ATSR GlobAER ATSR Swansey ATSR GlobAER filtered MISR vers.22

  39. AOD ATSR GlobAER filtered - total error

  40. AOD ATSR GlobAERfiltered - bias error

  41. AOD ATSR GlobAER filtered - spatial error

  42. AOD summary • ATSR products • ATSR is most promising (merged is poorer!) • ATSR AOD of GLOBAER • poorer than the ATSR AOD by Swansey • much poorer than MODIS or MISR • filtered version scores poorer mainly due to deterioation in spatial and temp. variability • ATSR AOD by GlobAER with filter • bias error is reasonable • spatial variability is poor • seasonality is poor

  43. extras • score differences … for overall errors • neg. difference  smaller error • pos. difference  larger error • gaa2 vs atsr • gaat vs atsr • gaa2 vs gaat • gaat vs misr

  44. gaa2 (filtered) vs atsr (swansey) black  better worse  green

  45. gaat (globaer) vs atsr (swansey) black  better worse  green

  46. gaa2 (filtered) vs. gaat (globaer) • b black  better worse  green

  47. gaat (globaer) vs. misr  better worse 

  48. AOD ATSR – GlobAER 2004

  49. AOD SEVIRI – GlobAER 2004

  50. AOD MERIS – GlobAER 2004

More Related