1 / 21

Paul Biemer RTI International and University of North Carolina Andy Peytchev RTI International

Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties. Paul Biemer RTI International and University of North Carolina Andy Peytchev RTI International. Estimating the Population Mean in an RDD Survey. TOTAL SAMPLE. Respondents { R }.

lola
Download Presentation

Paul Biemer RTI International and University of North Carolina Andy Peytchev RTI International

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding:An Evaluation of Error Properties Paul Biemer RTI International and University of North Carolina Andy Peytchev RTI International

  2. Estimating the Population Mean in an RDD Survey TOTAL SAMPLE Respondents {R} Nonrespondents Nonrespondents {NR} mean of NRs is unknown

  3. Methods for Adjusting and Evaluating for RDD Nonresponse • Very limited information on NRs in RDD surveys • Post-stratification adjustments are the norm • Effectiveness at reducing bias is questionable at best • Bias is sometimes evaluated using • Maximum followup effort approaches • Only evaluates reduction in bias due to slight elevations in response rates • Comparison to external gold standard estimates • Limited in scope to a few characteristics • Census block group geocoding • Error properties are largely unknown The focus of this research

  4. Census Geocoding (CG) Method • Obtain the addresses for nonrespondents (50-60% “success” rate) • Geocode addresses • Link to census aggregate data • Address matched: link unit to census block group (CBG) via geocode • Exchange matched: link to census tract (CT) via telephone number • Substitute the corresponding CBG or CT mean for the nonrespondent’s characteristic

  5. Estimating the Population Mean in an RDD Survey TOTAL SAMPLE Respondents {R} Nonrespondents Nonrespondents {NR} mean of NRs is unknown

  6. Impute Nonrespondent Characteristics from Census Aggregate Data TOTAL SAMPLE Respondents {R} Nonrespondents Nonrespondents {NR} Obtained from NRs CBG or CT

  7. Questions Addressed by this Research • What is the bias in ? • Is a valid estimate of the bias in the unadjusted (or post-stratified) estimator of the mean? • Does the CG method provide useful data for modeling response propensity? • The first two questions will be addressed in today’s presentation.

  8. Decomposition of the Bias in TOTAL SAMPLE Respondents Nonrespondents Correctly matched addresses Incorrectly matched addresses Correctly matched exchanges Incorrectly matched exchanges Size Expected Difference

  9. Components of the Bias in where

  10. Components of the Bias in where correct CBG match incorrect CBG match correct CT match incorrect CT match

  11. Estimation of the Bias Components • National Comorbidity Survey Replication (NCS-R) • National probability sample of 18+ in households • Face to face survey with 71% response rate • All addresses were geocoded • CG was applied to 8,178 responding hh’s that provided a telephone number (88% of NCS sample) • CG bias components estimated based on 41% response rate (response after 3 callbacks) • Sensitivity analysis based on three response rates: • 2 callbacks  26% response rate • 3 callbacks  40% response rate • 5 callbacks  60% response rate

  12. Why is it reasonable to use a face to face survey to evaluate the CG bias in an RDD survey? • The nonresponse mechanism is not a critical factor in the assessment of the CG bias. • A survey with a relatively high response rate is needed to evaluate the bias. • Addresses are known for all sample members and can therefore be geocoded to their correct CGs. • Sensitivity analysis can be performed to assess the effect on CG bias of increasing response rates.

  13. Weighted Respondent Mean, True Mean, and CG Imputed Mean for Available Characteristics

  14. Weighted Respondent Mean, True Mean, and CG Imputed Mean for Available Characteristics

  15. by Response Rate

  16. Average Estimates of for {s} = {CA}, {IA}, {CE}, and {IE} (Percentage points) Bias Component

  17. Average Relative Size of the Bias Components

  18. Conclusions • Bias in the CG estimates of NR bias is unacceptably large • race, age, and income were the most biased • Major source of bias • {IE} followed by {CA} (surprisingly) • Approximately 75% of the cases fall into these subsets • Correctly matching to CBGs reduces the bias, but minimally • Biases tend to build across components rather than netting out. • Increasing the survey response rate reduces bias in the CG approach; relative importance of each component is stable

  19. Next Steps • Further characterize the CG bias by its components • Consider the use of CBG and CT information obtain from the CG method for: • modeling of response propensities • adjusting for nonresponse bias

  20. Email me to request full reportppb@RTI.org

More Related