1 / 20

Measuring the quality of regional estimates from the ABS

Measuring the quality of regional estimates from the ABS. Jennie Davies and Daniel Ayoubkhani. Overview. ABS User needs ABS estimation Methods Results Recommendations Outcomes. Annual Business Survey (ABS). ONS’s largest business survey Sample of ~73,000 UK businesses

sebastian-mathis
Download Presentation

Measuring the quality of regional estimates from the ABS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring the quality of regional estimates from the ABS Jennie Davies and Daniel Ayoubkhani

  2. Overview • ABS • User needs • ABS estimation • Methods • Results • Recommendations • Outcomes

  3. Annual Business Survey (ABS) • ONS’s largest business survey • Sample of ~73,000 UK businesses • Covers agriculture (part), production, construction, distribution and service (part) sectors • Variables (collected and derived) include: • Turnover • Purchases of goods and services • Approximate Gross Value Added • Net capital expenditure

  4. Annual Business Survey (ABS) • National publication (November and June) • 4-digit SIC breakdown • Regional publication (July) • 12 UK regions, 2-digit SIC breakdown • Special Analysis system • Ad-hoc requests for lower level estimates (eg local authority, 3-digit SIC by employment sizeband etc) • Standard errors and CVs provided with national estimates but not regional

  5. User needs • Large range of ABS users • Central government, local government and devolved administrations, Eurostat, National Accounts, academia, consultancy firms, media, public • Quality measures therefore important for sound decision making • Lack of standard errors for regional estimates makes it difficult for users to assess accuracy

  6. User needs • Recent UKSA Assessment of ABS: • “there is insufficient information about methods and quality” • “there is no information about the resulting quality of the statistics and no caveats around their use” • QIF project therefore undertaken to develop methodology for calculating standard errors of published regional ABS estimates • Aim to provide users with information about the quality of regional estimates

  7. ABS estimation Ratio estimation National standard errors Reporting unit data National estimates GES Regional apportionment model Ratio estimation Modelled local unit data Regional estimates Regional standard errors ??? Small area estimation for small domains

  8. Standard error estimation (1) • Assume that the apportioned values are “true” returns • Single stage cluster sampling • Use GES to calculate standard errors

  9. Standard error estimation (2) • The regional apportionment model parameters depend on the sample data so are variable • Use bootstrapping to capture the use of the regional apportionment model

  10. Bootstrapping • Standard errors capture sampling variability • ie how much estimates vary under different possible samples • Bootstrapping re-samples from the original sample to create a new sample • Carry out estimation on new sample • Repeat lots of times • Calculate standard error of the resulting estimates

  11. Bootstrapping • Fix the model parameters based on the full sample data • quality assure method against GES • Re-fit the regional apportionment model in each iteration • include (possible) additional variance from the model

  12. Results • Compared the methods in terms of: • Differences in standard errors • Practical considerations • Results for turnover presented • Other variables produced similar results

  13. GES vs bootstrap without re-fitting the model • Bootstrapping without re-fitting the regional apportionment model in each iteration should be comparable with estimates from GES

  14. Bootstrapping with and without re-fitting the model • Comparing these to see if the regional apportionment model increases variances • Compared to bootstrap rather than GES to remove additional differences seen before

  15. Practical implications • Bootstrapping took ~30 hours • Relied on exact replication of the regional apportionment model • Problem for derived variables such as aGVA

  16. Recommendations • Use GES to produce standard errors • With caveats that the regional apportionment model is assumed to be fixed

  17. Outcomes • Report published on ONS website (Feb 2014) • Methodology approved by ABS Survey Management Board • Standard errors of regional estimates to be published for first time in July 2014 • Another QIF bid submitted to investigate standard errors of small area ABS estimates • Would allow for quality measures to be published alongside majority of Special Analysis requests

  18. Summary • ABS • User needs • ABS estimation • Methods • Results • Recommendations • Outcomes

More Related