Local and Urban Models: Challenges and Statistics

1. Comments on the DELTA tool statistics relating to local and urban models David Carruthers & Jenny Stocker FAIRMODE 4th Plenary and Working Group meeting 14th -16th June 2011 Swedish Meteorological and Hydrological Institute Norrköping, Sweden

2. Contents • What do we mean by ‘local’ and ‘urban’ models? • Challenges in predicting local concentrations • Influence of ‘background’ concentrations • Influence of averaging times • Comments on the NO2 statistics used by the DELTA tool • Comment on data capture

3. What do we mean by ‘local’ and ‘urban’ models? Hourly estimates of ‘background’ concentrations

4. What do we mean by ‘local’ and ‘urban’ models? Clearly there are different challenges for models that are grid based to those the resolve spatially to within tens of metres

5. Challenges in predicting local concentrations • Plots show modelled against observed concentrations for the AQD NO2 24-hourly maximum statistic • It is complicated to model the local effects – further away concentrations are made up of elements from different sources, inaccuracies in emissions etc are smoothed out. Roadside Urban background Kerbside Note different scales

6. Challenges in predicting local concentrations • Plots show modelled against observed concentrations for the AQD NO2 24-hourly maximum statistic • It is complicated to model the local effects – further away concentrations are made up of elements from different sources, inaccuracies in emissions etc are smoothed out. • Should the performance criteria for local model predictions be less strict than for the urban and regional models?

7. Influence of ‘background’ concentrations • Background concentration data taken from ADMS-Urban model input file (rural measurements) for London, 2008 • Variances are for observed concentrations and relate to the DELTA tool interpretation of the AQD statistics Magnitude of the variance depends heavily on the site type i.e. kerbside, urban background etc

8. Influence of ‘background’ concentrations • The DELTA tool indicates that ADMS performs better for PM10 and Ozone than it does for NO2. Related to the background concentrations. Modelled NOx concentrations Modelled PM10 concentrations

9. Influence of ‘background’ concentrations • The DELTA tool indicates that ADMS performs better for PM10 and Ozone than it does for NO2. Related to the background concentrations; are there other reasons? DELTA target plot for NO2 DELTA target plot for PM10

10. Influence of averaging times • Models predict the ensemble mean. In reality, pollutant concentrations are subject to fluctuations in time and space. Example quantile-quantile plot for hourly NO2 concentrations Models cannot be expected to predict the higher end of the observations, although an attempt at predicting the likelihood of these values is possible

11. Influence of averaging times • Models predict the ensemble mean. In reality, pollutant concentrations are subject to fluctuations in time and space. • Average statistical measures such as the daily average PM10 and 8-hour running mean for Ozone smooth out the effect of fluctuations, which makes these measures easier to model. • Another reason for criteria for NO2 be less strict compared to the other pollutants?

12. DELTA tool criteria and some statistics N = number of values; Oi/ Miobserved / modelled concentrations; = standard deviation of observations. • Target • RDE • RPE LV = Limit value; OLV closest observation to LV; MLV corresponding ranked modelled value. p = percentile corresponding to the allowed number of exceedences of the limit value..

13. Target • The DELTA tool indicates that ADMS performs better for PM10 and Ozone than it does for NO2. But clearly this is related to the background concentrations. DELTA target plot for NO2 DELTA target plot for PM10

14. Comment on NO2 statistics – target standard deviation: daily max or hourly • Normal distribution does not fits the observations well. • Skewed distribution would fit the hourly data better Normal distribution

15. Comments on the NO2 statisticsRPE and RDE • The DELTA tool interprets the NO2 EU air quality objective: ‘18 exceedences of 200 µg/m³ allowed per year’ (EUstat – calculated in Excel) by analysing statistics relating to the ‘1-hr daily maximum concentration’ (DTstat – calculated in DELTA) • It seems harder to achieve the criteria and goals using the latter statistic compared to the former. For example:

16. DELTA tool criteria and some statistics N = number of values; Oi/ Miobserved / modelled concentrations; = standard deviation of observations. • Target • RDE • RPE LV = Limit value; OLV closest observation to LV; MLV corresponding ranked modelled value. p = percentile corresponding to the allowed number of exceedences of the limit value..

17. Comment on data capture • In similar inter-model comparison studies within the UK (quote), minimal percentage data capture criteria have been applied, as monitors with poor data capture are less likely to be reliable. • Should the DELTA tool impose a minimum data capture criteria of, for example, 75%?

18. Any questions?

19. Influence of ‘background’ concentrations • The DELTA tool indicates that ADMS performs better for PM10 and Ozone than it does for NO2. • If we believe the input data for different pollutants to be of similar accuracy, we would expect the model output to be of similar order of accuracy (neglecting chemistry). • Why is the target for NO2 predictions so much harder to achieve for ADMS? • Is it related to ADMS being a local model? • Or is the NO2 statistic harder to achieve for all models?

20. Comments on the NO2 statistic • The DELTA tool interprets the NO2 EU air quality objective: ‘18 exceedences of 200 µg/m³ allowed per year’ (EUstat – calculated in Excel) by analysing statistics relating to the ‘1-hr daily maximum concentration’ (DTstat – calculated in DELTA) • It seems harder to achieve the criteria and goals using the latter statistic compared to the former. • Is there any way of using a stricter interpretation of the EU air quality directive in the tool?