Estimation and Weighting Part II

1. Estimation and Weighting Part II

2. NonresponseAdjustment (NR) Not all sampled HHs respond to surveys Ignoring nonresponse results in serious bias Simple weighted estimate x of a total X will fall short if baseweights are used in a simple weighted estimate only for respondents

3. Nonresponse Adjustment (NR) It is a HH level adjustment Adjust weights to account for in-scope (eligible) households not interviewed • Eligible HH should be interviewed • Some nonresponses are eligible • Some nonresponses are NOT eligible • Destroyed Housing Unit • Vacant Housing Unit • HH moved or everyone is deceased Survey procedures should identify 100% of sampled HH that are NOT eligible

4. Simple Nonresponse Adjustment Simple Random Sample n from N of households with baseweightbw = N/n • r household responses • nre eligible HH nonresponses • nrienonresponses that are NOT eligible HH N includes NOT eligibles also so naive nonresponse adjustment of n/r is wrong Use (r +nre)/r

5. Simple Nonresponse Adjustment • All persons have same basic weight wi = bw = N/n • New weights adjusted for nonresponse wi = bw*(r +nre)/r = (N/n)*(r +nre)/r • Use wi for a simple weighted estimate

6. Nonresponse Weighting Classes The simple adjustment assumes that nonresponse is the same everywhere Instead divide into weighting classes • N and n must be known for each weighting class • eligibility of nonresponses must be known also Make the same type of nonresponse adjustment restricted to a weighting class

7. Nonresponse Weighting Classes Possibilities for defining weighting classes • Each major geographic stratum • Do cities, villages, and rural areas have different nonresponse rates? • Do wealthy areas have different nonresponse rates than poor areas?

8. Ex: Weighting Class NR Adjustment • State A: N =500,000, n=2000, bw=N/n=250 r=1,800 nre=150 nrei=50 nraA = (r +nre)/r = (1,950/1,800) = 1.083 wi = bw * nraA= 250*(1.083) = 270.833 • State B: N =175,000, n=1750, bw=N/n=100 r=1000 nre=700 nrei=50 nraB = (r +nre)/r = (1,700/1,000) = 1.7 wi = bw * nraB= 100*(1.7) = 170

9. Ex: Weighting Class NR Adjustment • For State A the simple weighted estimate of employment becomes • Without nonresponse adjustment, the estimate of 750,000 was probably too low

10. NR Adjustment & Bias Weighting class adjustment for nonresponse does not eliminate nonresponsebias It assumes that respondents in a weighting class resemble the eligible nonrespondents • never exactly true Note the extreme difference in the example of the nonresponse adjustments • State A, 1.083 State B, 1.7 • A combined adjustment would suit neither state

11. Population Controls Most major household surveys benchmark to independent population controls • Population Controlling The controls are typically derived from census information updated to the present The “population” can be of HHs, of adults, of working-age adults, or of all persons One population control per geographic stratum is the simplest case • Some countries have complex multi-step “raking”

12. Population Controlling Purpose: Adjusts for undercoverage/overcoverage • perhaps due to frame deficiencies • perhaps due to nonresponse Lowers variances of data items correlated with the population controls • Ex: Employment is “large” and is highly correlated with the adult population

13. Population Controlling • This is a person-level adjustment to weights • Persons in the same HH get different adjustments to weights • Person weights after nonresponse adjustment are used to estimate populations • Estimates of populations are forced to equal “known” population controls • Coverage bias is les- senedbut not eliminated

14. Example of Population Controlling Continue the state A example where, through nonresponse adjustment: • all persons have wi = 270.833 • 3,000 persons responding as employed (EMP) • simple weighted estimate of employed 812,500 Expand the example • response for adult men: 2,100 with 1,700 EMP • Response for adult women 2,200 with 1,300 EMP

15. Example of Population Controlling Simple weighted estimates through nonresponse adjustment • adult men 2,100 * 270.833 = 568,750 • EMP men 1,700 * 270.833 = 460,417 • adult women 2,200 * 270.833 = 595,833 • EMP women 1,300 * 270.833 = 352,083 Suppose the population controls are: • adult men: 580,000 • adult women: 585,000

16. Example of Population Controlling • The assumption is that the population controls are batter than the estimates made from the sample • Men are somewhat undercovered either due to frame deficiencies, failure of households to properly report some men, or to nonresponse properties of households with men and nonresponse adjustment process. • Women are are somewhat overcovered • We want to change the sample weights so that sample estimates of the number of adult men and adult women match the controls

17. Example of Population Controlling Simple ratio adjustment to weights for men • adult men: adj = 1.01978 = 580,000/568,750 • adult women adj = .98182 = 585,000/595,833 Adjusted weights are different by gender • adult men: wi = 276.1905 = 270.833 * 1.01978 • adult women: wi = 265.9092 = 270.833 * .98182

18. Example of Population Controlling The populations “change” and so do the estimates of employed • adult men are increased • Adult women are decreased Simple weighted estimates after and before population controlling after before • adult men 2,100 * 276.1905 = 580,000 568,750 • EMP men 1,700 * 276.1905 = 469,524 460,417 • adult women 2,200 * 265.9092 = 585,000 595,833 • EMP women 1,300 * 265.9092 = 345,681 352,083 • EMP total 815,205 812,500

19. Review of Typical Estimation Steps To recap, household surveys have four estimation steps • Editing and Imputation are aimed at controlling response errors • Basic Weighting using probabilities of selection would produce essentially unbiased estimates if 100% response with no response error was possible • Weighting Class Nonresponse Adjustment helps avoid some obvious biases that arise when nonrespondents are ignored • Population Controls help minimize some coverage problems and reduce variances