Managing non-response in the Russian and in East European context

1. Managing non-response in the Russian and in East European context Oxana Sinyavskaya, Independent Institute for Social Policy (Moscow) Generations and Gender Survey (GGS) in Russia Project Director MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012

2. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Outline • Response rates levels • Structure of non-response • Strategies of estimating non-response bias • Overview of methods used to increase response rate

3. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Response rates levels In Russia and some other Eastern European countries

4. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Response rates in ESS-2010

5. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Response rates in ESS, 2004-2010, selected countries

6. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Response rates in selected countries: GGS, 1st wave

7. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Dynamics of response rates in Russia, 1990s

8. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Dynamics of response rates in Russia, 2000s

9. Independent Institute for Social Policy (IISP): www.socpol.ru GGS Sample in Russia • 11 261 respondents, 18-79 years old • from 32 regions of Russia • 1 respondent – 1 household • Compatibility with the Russian Longitudinal Monitoring Survey sample • Representative at the national level

10. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Response rates in RusGGS-2004 by settlements

11. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Response rates in Russia and its ‘capitals’

12. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Structure of non-response In Russia

13. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Break down of response and non-response in different surveys

14. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Response rates and eligibility of households in RusGGS-2004

15. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Proportions of not-contacted and refused households by settlements in RusGGS-2004

16. GGP Panel Maintenance Workshop 13/07/2010 Reasons of not-response in RusGGS-2007 • (Both panel and new respondents 21-82 together) • No contact – 41.8% • Refusal – 35.3% • Moved between 2 waves – 8.3% • Death – 4.6% • Other reasons – 7.7%

17. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Structure of refusals by settlements in RusGGS-2004

18. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Factors increasing non-response (1) • Urbanization / density of population: the worst – “capitals” (Moscow, St. Petersburg) • Criminal situation • Season – Autumn & Spring = best; Summer = worst • Pre-election time • Sample frame: e.g. when electoral (voting) lists were last updated; no reliable official lists of households in urban area; no unified register of households; no individual register available

19. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Factors increasing non-response (2) • Survey topic • Length of interview • Number of interviews per interviewer • Expertise of interviewers • Motivation of interviewers • Motivation of respondents • Control over interviewers

20. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Strategies of estimating non-response bias Used in Russian GGS

21. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Strategies of estimating non-response bias • Not discussed in the literature • Used in GGS: • Comparison of respondents’ characteristics with statistics data (Census 2002) • Analysis of panel attrition

22. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Comparison of RusGGS-2004 with Census 2002: males

23. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Comparison of RusGGS-2004 with Census 2002: females

24. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 GGS panel: response and attrition rates vary across settlements

25. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Regional variation of attrition in Russian GGS panel (2004 and 2007) • Among 32 regions in the GGS sample: • 2 had attrition rate between 4-5% • In 5 regions – attrition – between 10-20% • 11 regions – between 20-30% • 10 regions – 30-40% • 4 regions – from 40 to 86% • 4 worst regions: • Primorskiy krai (Far East) – attrition – 86.3% • Moscow city – 54.5% • St. Petersburg – 53.9% • Moscow oblast (region) – 46.5%

26. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 RusGGS: Socio-demographic features of attrition between 2004 and 2007

27. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Methods used to increase response rate In Russia

28. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Increasing response rates in fieldworks • Substantial number of contacts and appropriate schedule of contacts: e.g., at least 4 in ESS, at least 3 in GGS; some contacts should be in the evening time and on weekend • Re-contact respondents after some time after 1st refusal • Motivation of interviewers, e.g. (from ESS): • bonus for successful interview after 4 attempts of contacts • bonus for response rater higher than 70% • higher payment for interview if refusal conversion has occurred • Motivation of respondents • Conditional monetary / non-monetary incentives upon completion of the interview • Regular control over interviewers work (emphasized by SPRILO team)

29. MANAGING PROBLEMS WITH MISSING DATA IN SURVEY RESEARCH, September 10, 2012 Questions? Discussion? Email: o.sinyavskaya@socpol.ru