On the Verification of Particulate Matter Simulated by the NOAA-EPA Air Quality Forecast System

1. On the Verification of Particulate Matter Simulated by the NOAA-EPA Air Quality Forecast System Ho-Chun Huang1, Pius Lee1, Binbin Zhou1, Jian Zeng6, Marina Tsidulko1, You-Hua Tang1, Jeff McQueen3, Qiang Zhao7, Shobha Kondragunta2, Rohit Mathur4, Jon Pleim4, George Pouliot4, Geoff DiMego3, Ken Schere4, and Paula Davidson5 1 Scientific Applications International Corporation, Camp Springs, Maryland. 2 NOAA/NESDIS Center for Satellite Applications and Research, Camp Springs, Maryland. 3 National Centers for Environmental Prediction, Camp Springs, Maryland. 4 National Oceanic and Atmospheric Administration, Research Triangle Park, NC. (On assignment to the National Exposure Research Laboratory, US EPA) 5 Office of Science and Technology, National Weather Service, Silver Spring, MD. 6 Earth Resources Technology Inc., Annapolis Junction, MD. 7I.M. Systems Group, Inc., Rockville, MD.

2. Outline • NOAA-EPA Air Quality Forecast System • GOES and AQF atmospheric optical depth (AOD) • NCEP verification results • Summary

3. Air Quality Forecast System • CONUS (ozone) became operational model on September 18, 2007 • Developmental model; operational* + PM Chemistry • CMAQ v4.5 driven by the WRF/NMM at 12 km • NEI (2001), BEIS3, Mobile 6 • AERO3: Aerosol Module with SOA (no sea salt) • Updated ISORROPIA for numerical stability at low relative humidity • Euler Backward Iterative (EBI) Solver for CB4 • Minimum Kz to mimic urban island

4. AOD Comparisons • In-site measurement (AERONET, AIRS) (Marina Tsidulko) • Satellite measurement – GOES product comparisons with AERONET and MODIS (Prados et al, 2007) • (AERO) good for AOD > 0.15, Negative bias for AOD > 0.35 • (MODIS) good agreement and correlation of high AOD • CMAQ AOD comparison with IMPROVE, MODIS, and AERONET in the eastern US (Roy et al, 2007) • good spatial and temporal patterns • CMAQ AOD is often less than MODIS AOD for the same concentration

5. The NCEP/EMC Real-time*AOD Verification • AQF AOD: The column integration of extinction (σ) due to particulate scattering and absorption and layer thicknesses (ΔZi) • AQF AOD vs.GOESAOD • Frequency: Daily (April to September 2007) • Data: hourly from 1215 – 2115 UTC • Domains: CONUS, East US, and West US

6. The GOES Derived AOD (Prados et al, 2007) Visible Infrared

7. AQF modeling and verification domains

8. Null GOES AOD mean over the period Total : 66.6% Cloud : 41.8% White Noise : 24.8%

9. mean over the period Total : 55.0% Cloud : 46.4% White Noise : 8.6% mean over the period Total : 78.3% Cloud : 34.8% White Noise : 43.5%

10. The NCEP/EMC Real-time*AOD Verification • Thresholds • <0.1, >0.1, >0.2, >0.3, >0.4, >0.5, >0.6, >0.8, >1.0, >1.5, and > 2.0 • Skill Scores • Critical Success Index (Threat Score; CSI) • Probability of Detection (POD) • False Alarm Ratio (FAR) • #_of_Fcst / #_of_Obsv (BIAS) • Equitable Threat Score (ETS) • Accuracy rate • Type of figures • Daily average time series (per month) • Daily average by threshold • Monthly average by threshold

11. http://www.emc.ncep.noaa.gov/mmb/aq/

12. http://www.emc.ncep.noaa.gov/mmb/hchuang/web/html/score_mon.htmlhttp://www.emc.ncep.noaa.gov/mmb/hchuang/web/html/score_mon.html

14. Observed YES NO YES a b Forecast N=a+b+c+d NO c d d O=a+c c F=a+b H=a a b Bias = F/O = (a+b)/(a+c) CSI = H/(F+O-H) = a/(a+b+c) POD = H/O = a/(a+c) False Alarm ratio = 1-H/F = b/(a+b) Accuracy rate = (N-F-O+2H)/N = (a+d)/(a+b+c+d)

15. d O=a+c c F=a+b H=a a b < 0.1 Bias : number of points > 0.4 > 0.1 > 0.5 > 0.2 > 0.6 > 0.3 > 0.8

16. d O=a+c c F=a+b H=a a b Probability of Detection < 0.1 > 0.4 > 0.1 > 0.5 > 0.2 > 0.6 > 0.3 > 0.8

17. AQF does not account foradditional particulate sources? • Inventory wild fire emissions, not real-time data • Sea Salt • Long range transport of dust, aerosol, and chemical species across modeling boundary

18. > 0.1 Critical Success Index d > 0.2 O=a+c c F=a+b H=a a b > 0.3

19. X X

20. Pearson Correlation Coefficientbetween the AQF skill score (CSI) and the number of Null GOES data due to Cloud TOTAL DAYS = 183 CSI CLUD CONUS > 0.1 NUM 167 r= -0.3165 r2 = 0.1002 t = -4.2852 CSI CLUD CONUS > 0.2 NUM 167 r= -0.2978 r2 = 0.0887 t = -4.0075 CSI CLUD CONUS > 0.3 NUM 167 r= -0.2595 r2 = 0.0673 t = -3.4512 CSI CLUD E US > 0.1 NUM 167 r= -0.3580 r2 = 0.1282 t = -4.9254 CSI CLUD E US > 0.2 NUM 167 r= -0.2774 r2 = 0.0769 t = -3.7087 CSI CLUD E US > 0.3 NUM 167 r= -0.2462 r2 = 0.0606 t = -3.2630 CSI CLUD W US > 0.1 NUM 167 r= -0.0520 r2 = 0.0027 t = -0.6686 CSI CLUD W US > 0.2 NUM 167 r= 0.1309 r2 = 0.0171 t = 1.6958 CSI CLUD W US > 0.3 NUM 167 r= 0.1286 r2 = 0.0165 t = 1.6661

21. August 2-5, 2007

22. August 15, 2007

23. SUMMARY • Good spatial AQF PM distribution with low bias on the AOD  unresolved PM sources or processes • It is difficult to access the AQF PM skill in the western US due to strong surface reflectivity • Negative correlation (in the eastern US) between the AQF PM skill score and null satellite AOD because of cloud (clear sky  better skill score) was observed • Further investigation is needed to understand the (non-linear) relationship between cloudiness and AQF PM skill, as well as the processes that impact AQF PM skill

24. http://www.emc.ncep.noaa.gov/mmb/aq/