1 / 16

Realising Value from Follow-up of Non-response in ABS Business Surveys

Realising Value from Follow-up of Non-response in ABS Business Surveys. Authors: Matt Ashton, Louise Gates, Dina Neiger, Greg Griffiths Presenter: Greg Griffiths. Total Survey Design.

collice-lindsay
Download Presentation

Realising Value from Follow-up of Non-response in ABS Business Surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Realising Value from Follow-up of Non-response in ABS Business Surveys Authors: Matt Ashton, Louise Gates, Dina Neiger, Greg Griffiths Presenter: Greg Griffiths

  2. Total Survey Design • Linacre & Trewin (JOS 1993) : A key question for a NSO is to determine how available funds are to be allocated over the full range of activity required. • This requires a full understanding of fitness for purpose, of how the quality of collection outputs change due to increasing effort and an understanding of process and error interdependencies.

  3. Practising Total Survey Design • Need to further our knowledge of the survey process and the quality/cost interactions between its component activities. • Within the ABS the Establishment of the Economic Statistics Data Centre has provided an opportunity to : • Separate the design of field operations from sample design and data compilation • Establish systems for the collection of paradata (data about process) • Collection of data on current processes is not enough however, there needs to be a good understanding of the costs of alternative processes. • A consequence of process complexity and the developing state of our knowledge is that we are still frequently grappling with developing our understanding of component sub-processes.

  4. Intensive Follow-up (IFU) Standard follow-up procedures for ABS business surveys include: • Reminder letters; • Phase 1 IFU: an initial phone contact for all outstanding units starting at a fixed time after the due date to determine the live/dead status of the unit, to confirm that they received the form and to ascertain a date for the form’s return; • Phase 2 IFU: phone follow-up targeted at significant units. During this phase all outstanding units are likely to be contacted, however, unlike Phase 1, the contact is prioritised so that significant units are contacted first; • Phase 3 IFU: phone follow-up of critical non-respondents ; • Ongoing follow-up action from direct contacts and in-coming calls.

  5. Current ABS IFU practice Significance of units determined in many different ways within and across collections: • strata with 'low' response rates or high contribution from imputation • units with specific business structures or activity, high complexity or special processing requirements • units according to their contribution to estimate • units due to their history (past non-response or new-on) • consequential contact of units with common ownership • strata based on sample design properties (units in completely enumerated strata, units whose responses will or will not be augmented by administrative data) • strata based on potential change in variance • units based on some definition of unit 'size' eg unit sample weight compared to total weight at some level of aggregation. • non respondents in publication cells with very low target sample numbers (for example to alleviate potential confidentiality issues in final data) • units with high weighted expected absolute difference between imputation and response (McKenzie ICES II).

  6. ABS Projects under way • To understand from the viewpoint of statistical theory how units should be prioritized so as to minimize mean square error and develop unit scores. • To identify best practice and cost-efficient approaches to cost-effective collection of data for the economic statistics program • To understand relationship between cost of follow-up and survey response & examination of telephony patterns and processes to identify cost effective practice

  7. Number of Outbound contactsSingle collection cycle for Annual Business Collection

  8. Number of Outbound contactsSingle collection cycle for Quarterly Business Collection

  9. All outbound contacts by type of contactAnnual and Quarterly Survey compared

  10. Percentage contribution to estimates by number of contacts- Annual survey – one cycle

  11. Percentage contribution to estimates by number of contacts - Quarterly survey – one cycle

  12. Time slice dataQuarterly survey – one cycle

  13. Time slice dataQuarterly survey – one cycle

  14. Hourly variation from mean contact success rate

  15. Future work on contact data • further surveys and further work on the contact patterns • impact of reminder letters • impact of pre-approach letters • impact of significance flag • difference between faxed and mailed reminders • Experiment with intervening phone call between first and second reminders

  16. Wrap up • We are still at the stage of trying to understand components of the overall data gathering and dissemination process. • Need to ensure systems and data stores are designed with the collection of paradata in mind, • Paradata collection for any one process be done with a view to upstream & downstream processes. • Cost/Effort versus Benefit models are still a cottage industry and much analysis of paradata remains to be done.

More Related