Uncertainty Estimation

1. Uncertainty Estimation Simon Cousens and Richard Silverwood

2. Overview • an implementation/coding issue with the Loess approach • accounting for within-source correlation • increasing uncertainty with increasing sources(?)

3. Implementation issue • Murray at al. describe incorporation of two sources of uncertainty – random noise in data points through resampling parameter values using point estimates and variance-covariance matrix - (one aspect of) model uncertainty through use of a range of α values • Wardlaw subsequently noted surprisingly narrow uncertainty ranges for Belarus, Congo, Cote d’Ivoire, Ukraine

4. Example: Congo

5. Example: Belarus

6. Accounting for within source correlation • Loess and spline approaches have assumed independence of data points • In practice multiple data points used in analysis derived from a single source (e.g. DHS) and assumption of independence likely to be violated One approach is to fit a multi-level model. Loess model: log(yij) = β0j + β1xij +β2zij + eij where β0j = β0 + uj and uj ~ N(0,σu)

7. Results In general, little change in point estimates (e.g. 2010)

8. Results Uncertainty ranges within period of observed data increase (e.g. 1990)

9. Results Pattern less clear outside period of observed data (e.g. 2010)

10. Some specific country examples Costa Rica

11. Some specific country examples Sierra Leone

12. Increasing uncertainty with increasing sources? Eritrea

13. Eritrea

14. Eritrea

15. Comoros

16. The Gambia

17. Bangladesh

18. Bangladesh No obvious “asymtotic” pattern – is this useable in any way?