1 / 28

Building Wave Response Rates in a Longitudinal Survey

This article discusses the importance of achieving target response rates in a longitudinal survey and evaluates the quality of nonresponse adjustment strategies. It examines the impact on key survey estimates and discusses implications of alternative field strategies.

erwinm
Download Presentation

Building Wave Response Rates in a Longitudinal Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building Wave Response Rates in a Longitudinal Survey: Essential for Nonsampling Error Reduction or Last In - First Out? Steven B. Cohen Fred Rohde and William Yu Agency for Healthcare Research and Quality

  2. Purpose of Discussion • Need for essential longitudinal data on health care coverage, use and expenditures to inform health care policy and practice • Description of the Medical Expenditure Panel Survey (MEPS): purpose, longitudinal design and analytical capacity • Focus on field efforts to achieve target response rates

  3. Purpose of Discussion • Evaluations of the quality of the MEPS nonresponse adjustment strategies • Determination of characteristics for cases fielded at end of field period (EOF) & conversion of temporary refusals (TNR). • Examine ROI for inclusion of these cases.

  4. Purpose of Discussion • Examine impact on annual and longitudinal response rates; completion of self administered questionnaires (SAQ) • Impact on key survey estimates of health insurance coverage and expenditures • Implications of alternative field strategies

  5. Medical Expenditure Panel Survey (MEPS) Annual Survey of 15,000 households: provides national estimates of health care use, expenditures, insurance coverage, sources of payment, access to care and health care quality Permits studies of: • Distribution of expenditures and sources of payment • Role of demographics, family structure, insurance • Expenditures for specific conditions • Trends over time

  6. Key Features of MEPS-HC • Survey of U.S. civilian noninstitutionalized population • Sub-sample of respondents to the National Health Interview Survey (NHIS) • Linkage to NHIS • Oversample of minorities and other target groups • Panel Survey – new panel introduced each year • Continuous data collection over 2 ½ year period • 5 in-person interviews (CAPI) • Data from 1st year of new panel combined with data from 2nd year of previous panel

  7. MEPS Overlapping Panels(Panels 8 and 9) MEPS Household Component MEPS Panel 8 2003-2004 1/1/2003 1/1/2004 NHIS 2002 Round 1 Round 2 Round 3 Round 4 Round 5 NHIS 2003 Round 1 Round 2 Round 3 Round 4 Round 5 MEPS Panel 9 2004-2005

  8. MEPS Household ComponentSample Design Oversampling of policy relevant domains 1996 Minorities (Blacks & Hispanics) 1997 Minorities Low income Children with activity limitations Adults with functional limitations Predicted high expenditure cases Elderly 1998-2001 Minorities 2002+ Minorities, Asians, Low Income 15,000 households; ~35,000 persons

  9. Target Precision Specifications for national and regional estimates; policy relevant subgroups Overall Design effect of 1.6 200 PSU design (Max) Overall/round specific survey response rate requirements Linkage to NHIS Multistage design Disproportional sampling Longitudinal design Minimize survey cost for fixed precision Design Specifications

  10. MEPS, 1996–2006: Number of uninsured, under age 65 Number in millions Source: Center for Financing, Access, and Cost Trends, AHRQ, Household Component of the Medical Expenditure Panel Survey, 1996–2005 Full-Year and 1996–2006 Point-in-Time Files

  11. Trends in Concentration Percentage of expenditures Source: National Medical Care Expenditure Survey, 1977; National Medical Expenditure Survey, 1987; Medical Expenditure Panel Survey, 1996 and 2005.

  12. MEPS Field Force • Westat is data collection organization • 500 interviewers • Sample allocated in ~ 200 PSUs, spread across all 50 states • extensive training modules • Information on socio-demographic characteristics of households available based on linkage with NHIS • Remuneration for survey participation

  13. Tool Chest of Methods to Maximize Survey Response • Recruitment of experienced and bilingual interviewer • 10+ days training (including procedures for obtaining signed consents) • Uses of MEPS data as reference materials for interviewers • Periodic retraining and special trainings (e.g. methods to improve response rates) • Respondent remuneration • Advance mailings from co-sponsors of survey • Monthly planning calendar and MEPS DVD • Daily emails to interviewers regarding interviewing progress • Multiple contacts for refusal conversions

  14. MEPS Target Response Rates by Round and Overall *NHIS response rate among households designated for MEPS. Note: Year 1 and the Overall response rate include the NHIS response rate.

  15. Person Level (survey attrition) Nonresponse Adjustment Covariates • Factors associated with survey attrition (after R1) • Indicator for initial refusal to R1 interview • Family size • Age • MSA, census region • Marital status (family reference person) • Race/ethnicity • Education of reference person • Employment status • Health insurance status • Total expenditures (in yr 1 for yr 2 adj.) • # doctor visits (in yr 1) • Self reported health status

  16. Person Level Adjustments:Annual Estimates • Each MEPS panel weighted separately • Nonresponse adjustment for complete nonresponse and for survey attrition • Final Poststratification adjustment –CPS 12/31: age, race/ethnicity, sex, region, MSA status, poverty status

  17. Testing for Panel Effect

  18. Capacity of MEPS to Produce Comparable NHIS Estimates of Health Insurance Coverage

  19. Characteristics of Respondents Fielded at End of First Round or Temporary Refusal Initial Refusals: Higher likelihood: • MSA residence; Northeast region; white Non-Hispanic; elderly; excellent health; some high school; family size 2+; Attrite in future waves of data collection End of Field Period Cases Higher likelihood: • Race: Asian or Black • in excellent health • Attrite in future waves of data collection

  20. Testing for Reluctant Response Effect on Coverage Estimates ------------------------------------------------------- DF Wald F P-Value ------------------------------------------------------- OVERALL MODEL 22 107.98 0.0000 JULY INTERVIEW 1 2.38 0.1244 TEMP. REFUSALS 1 0.92 0.3393 SEX 1 98.02 <0.0001 RACE/ETHNICITY 3 58.42 <0.0001 MARITAL STATUS 4 16.90 <0.0001 EDUCATION 4 10.94 <0.0001 POVERTY STATUS 4 43.97 <0.0001 MSA STATUS 1 4.34 0.0382 INDIVIDUAL INCOME 1 35.52 <0.0001 MEDICAL $ 1 35.79 <0.0001 ------------------------------------------------------- -2 * Normalized Log-Likelihood Full Model: 13037.78 Pseudo R2: :0.194167

  21. Mean Number of Contacts by Month

  22. Mean Number of Temporary Refusals by Month

  23. Conditional Response Rates by Month of Round 1 Response: Panel 9

  24. Impact on MSE of Mean Medical Expenditure Estimates for alternative field strategies

  25. Impact on MSE of Proportion with Medical Expenses in excess of $10,000 for alternative field strategies

  26. Summary • Need for accurate and reliable national data on health insurance coverage to inform policy and practice • MEPS design features and analytical capacity • Statistical, methodological and operational design features to adjust for nonresponse and attrition • Evaluation of estimation strategies to correct for survey attrition • Examination of ROI for inclusion of difficult cases • Options identified for more efficient and effective field strategies

More Related