Understanding Response Models in Surveys

1. Chapter 3: Response models • Handbook: chapter 2 • Introduction • Fixed Response Model • Random Response Model • Effects on confidence intervals

2. Response Models • Fixed Response Model • The population consists of only two types of elements. • Respondents are in the response stratum. They always respond. • Nonrespondents are in the nonresponse stratum. They never respond. • Random Response Model • Each element has a certain probability to respond. • The response probability is different for each element. • The response probabilities are unknown.

3. Response Models • Fixed Response Model • Population divided into two strata • Response stratum: Persons in this stratum always respond. • Nonresponse stratum: Persons in this stratum never respond. • Response indicators R1, R2, …, RN, with • Rk = 1 if element k is in Response stratum • Rk = 0 if element k is in Nonresponse stratum Response stratum Nonresponse stratum

4. Response Models • Fixed Response Model • Simple random sample of size n from population. • Sample indicators a1, a2, …, aN, with • ak = 1 if element k is in the sample • ak = 0 if element k is not in the sample • Only nR elements in the response stratum are observed. • Question: Is the response mean a good estimator for the mean of the complete target population? Response stratum Nonresponse stratum

5. Fixed Response Model • Estimator • Expected value • Bias • The bias of the estimator is determined by • Differences (on average) between respondents and nonrespondents. • Relative size of nonresponse stratum, i.e. the expected nonresponse rate.

6. Fixed Response Model • Example • Dutch Housing Demand Survey 1981 • Target variable: Intention to move within two years • Information about nonresponse from follow-up survey • Estimate based on response: 29.7% • Estimate based on nonresponse: 12.8% • Contrast K = 16.9% • Relative size of nonresponse stratum Q = 0.288 • Bias of estimator = 16.9  0.288 = 4.9%

7. Response Models • Random Response Model • Each element k has an unknown response probability ρk • Response indicators R1, R2, …, RN, with • Rk = 1 if element k is in responds • Rk = 0 if element k does not respond • P(Rk = 1) = ρk, P(Rk = 0) = 1 – ρk • Sampling • Simple random sample of size n from population. • Sample indicators a1, a2, …, aN, with • ak = 1 if element k is in the sample • ak = 0 if element k is in in the sample • Number of respondents

8. Random Response Model • Estimator • Expected value • Bias • The bias of the estimator is determined by • Correlation RρY between response behaviour and target variable • Variation of response probabilities • Mean of response probabilities (expected response rate)

9. Random Response Model • Example • Estimating the mean income of working population of Samplonia. • Simulation 1: 1000 samples, size n = 40, no nonresponse • Simulation 2: 1000 samples, size n = 40, nonresponse increases with income • Simulation 3: 1000 samples, size n = 80, nonresponse increases with income

10. Effect of nonresponse on confidence interval • Full response • 95% confidence interval: • Confidence level • Nonresponse • Confidence interval • Confidence level • Confidence level not equal to 0.95.

11. Effect of nonresponse on confidence interval • Confidence level as a function of the relative bias

12. Effect of nonresponse on confidence interval • Example • Confidence intervals for the mean income of working population of Samplonia. • Simulation 1: 30 samples, size n = 40, no nonresponse. Confidence level = 97% • Simulation 2: 30 samples, size n = 40, nonresponse increases with income. Confidence level = 60%

13. Response models • Some conclusions • There is only a bias if the target variable of the survey is related to the response behaviour. • The magnitude of the bias of estimators is only partly determined by the response rate. • The magnitude of the bias of estimators is also determined by the variation in response probabilities. • In case of nonresponse, the confidence interval cannot be used any more as a measure of accuracy. • The nonresponse bias is not reduced by increasing the sample size.