The European Statistical Training Programme (ESTP)

1. The European Statistical Training Programme(ESTP)

2. Chapter 2: The nonresponse problem • Handbook: chapter 1 • What is nonresponse? • Causes of nonresponse • Response rates • Some international trends

3. Nonresponse • Nonresponse • Elements that are selected in the sample, and that are also eligible for the survey, do not provide the required information. • Two types of nonresponse: • Unit nonresponse: a selected element does not provide any information at all. • Item nonresponse: a selected element does answer some questions, but not all of them. • Why is nonresponse a problem? • Smaller sample size. • Nonresponse bias due to selective nonresponse.

4. Selective nonresponse • Example: Dutch Housing Demand Survey 1981 • Initial sample size: 82,849 persons. • Response: 71.2%. • Follow-up survey among nonrespondents: • Percentage of movers in response: 29.7%. • Percentage of movers in nonresponse: 12.8%.

5. Selective nonresponse in Dutch Surveys • Victimization Survey • People who are afraid to be home at night are less inclined to participate in the survey. • Mobility Survey • The more mobile people were harder to contact, and thus under-represented in the survey, • Labour Force Survey • Refusal rates are higher among the unemployed. • Election Survey • Participation rates are higher among voters. Voters are over-represented.

6. Selective nonresponse Simulation • Estimation of the mean income in Samplonia. • Simulate 1,000 samples of size 50. • Case 1: No nonresponse. • Case 2: Nonresponse increases with income. • Conclusion: estimates are systematically too low in case of nonresponse. • Confidence intervals are wrong. Selective nonresponse No nonresponse

7. Causes of nonresponse (CAPI) No Contact? No Eligible? Yes Nonresponse Non-contact Over-coverage No Eligible? Over-coverage Yes No Participates? Nonresponse Refusal Yes No Able? Nonresponse Not-able Yes Response

8. Causes of nonresponse • Non-contact • Nobody home, phone not answered. • Make more contact attempts (up to 6). • Refusal • Can be permanent or temporary. • Role of interviewers important. • Use of incentives? • Not-able • Physical or mental problems. • Language problems. • Unprocessed • Interviewers are unable to handle their workload.

9. Response rates • Definition Lynn et al. (2002) • Proportion of eligible elements in the sample for which a questionnaire has been completed: • Notation • nE = Number of eligible elements in the sample • nR = Number of eligible respondents • nE = nR + nNC + nRF + nNA • Initial sample size n = nE + nOC. • So nE = n – nOC. • Problem: over-coverage unknown for non-contacts.

10. Response rates • Estimating the response rate • Assumption 1: all non-contacts are eligible: • Assumption 2: the proportion of eligibles among non-contacts is the same as the proportion of eligible among contacts:

11. Response rates - Example • Fieldwork results for the Dutch Integrated Survey on Household Living Conditions in 1998: • If all non-contacts are eligible, the response rate equals • If the proportion of eligibles among contacts and non-contacts is the same, the response rate is

12. Response rates • Response percentages in some Dutch surveys

13. Response rates • Some other response rate problems • Self-administered surveys (no interviewers): • It is unclear who is completing the questionnaire. Is the person eligible? • The cause of nonresponse is unclear. • Unequal probability sampling: How to compute the response rate if the sample is selected with unequal probabilities? • Unweighted response rate as a quality indicator of the performance of interviewers. • Weighted response rate as a quality indicator of estimates. • Household survey: How to define response/nonresponse of a household?

14. International trends • De Heer (1999) compared Labor Force Surveys for official statistical offices • Large difference in response rates and trends • Rates vary from around 60 (NL) to around 90 (USA) percent. • UK, USA stable pattern. • Finland, Sweden downward trend. • Belgium seems to be recovering. • The Netherlands world champion nonresponse.

15. International trends • Also a large differences in the composition of nonresponse in non-contacts and refusals • UK: non-contact decreasing whereas in most countries it is increasing. • Refusals mostly differ in rate, not so much in trend.

16. International trends • Possible explanations towards differences: • Difference in general design

17. International trends • Stoop (2005) analyses data from the first round of the European Social Survey, in 2002/2003. In the European Social Survey: • One sampling design • Response target: 70%; non-contact target: 3% • Central fieldwork specifications • The results are very different: • The response rate varies from 33% (Switzerland) to 80% (Greece) • The non-contact rates varies from 2% (Poland) to 15% (France) • The refusal rate varies from 14% (Hungary) to 55% (Switzerland) • The Netherlands is doing well, with a response rate of 68%, non-contact of 3% and refusal rate of 24%.

18. International trends • Results of the first round of the ESS

19. Response rate in the European Social Survey in the first two rounds International trends • The second round from the ESS in 2004/2005 showed and improvement of low response countries with respect to the first round (Stoop and Billiet, 2007):

20. Response rate and refusal/ (refusal+noncontact) in the European Social Survey Round 2 International trends • Relationship between the response rate and the composition of the nonresponse. • Refusal is more important than non-contact in most countries • No relationship between response rate and response composition