Do Monetary Incentives Increase Business Survey Response Rates?

1. Do Monetary Incentives Increase Business Survey Response Rates? Results from a Large Scale Experiment Paul Biemer, RTI International and University of North Carolina Christopher Ellis, Angela Pitts and Kimberly Robbins RTI International

2. Outline of the Presentation • Essential information about the establishment survey • Design of the incentive experiment • Results • Conclusions

3. The O*NET Survey • Occupation Information Network Survey (O*NET) • Provides descriptive ratings on 800+ U.S. occupations • Target population is all U.S. employees in these 800+ occupations • Continuing survey since 1997 • Mode of interview • Telephone contact of establishments • Point of contact (POC) is usually Human Resources Manager • PAPI or Internet questionnaires completed by selected employees

4. The O*NET Survey (cont’d) • Two-stage sample design • Establishments and employees within establishments in selected occupations • ~24,000 establishments/year (79% RR) • ~36,000 employee respondents/year (69% RR)

5. O*NET Incentive System • All incentives are prepaid (prior to receipt of questionnaires) • For the establishment: • O*NET toolkit (personnel job description aid) • For POC: • Desk clock • Framed certificate of appreciation • $20 money order (to be tested) • For employees • $10 cash

6. POC Incentive Experiment: Motivation • Concern that POC incentives not commensurate with POC tasks • Some evidence of diminished POC effort during employee nonresponse follow-up phase • Establishment and employee nonresponse rates were low for some occupations • POC does not receive monetary incentive although employees do ($10)

7. Overview of Data Collection Protocol

8. Incentives sent here Brochure sent here Overview of Data Collection Protocol

9. Cover of the POC Incentive Brochure FRONT BACK

10. Inside of the POC Incentive Brochure

11. Experimental Design • Split sample design (75/25 split) • Incentive group: POCs offered $20 money order in addition to clock and framed certificate • Control group: POCs offered only clock and framed certificate • Money orders could be made out to POC, the business or a favorite charity • Interviewers (called business liaisons or BLs) handled both incentive and non-incentive cases

12. Sample Allocations to the Control and Incentive Groups

13. Results

14. Unweighted Response Rates (%)

15. Other Results • No effect on response rates by • size of establishment • industry grouping • urban or rural regions • occupation of employee • Some evidence of a negative effect for some sectors

16. Cumulative Employee Response Rates for by Week of Data Collection

17. Summary of Results • No evidence of an effect on establishment cooperation rates • Weak evidence of an effect on employee cooperation rates • Both negative and positive effects detected for small subgroups • Overall, no significant difference • No cost advantage • Follow-up costs were the same • Speed of response was the same • Incentive costs were not offset by efficiency gains

18. Interpreting the Findings • No establishment-level effect • $20 incentive viewed as trivial by POCs • Framed certificate and clock could have much larger perceived value • Decision to participate shared by POC and company • POC cannot accept money in many establishments • No effect at the employee-level • POC has limited ability to affect employee response • $20 ineffective for increasing follow-up intensity