1 / 37

Diary non-response and quality in 2 surveys

Diary non-response and quality in 2 surveys. Delphine Roy, INSEE (France). INTRODUCTION. Diaries as a reliable data collection instrument Gershuny (2000) compares diary information with retrospective questionnaire: memory effect, loss of information

hakeem-dillon
Download Presentation

Diary non-response and quality in 2 surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diary non-response and quality in 2 surveys Delphine Roy, INSEE (France)

  2. INTRODUCTION • Diaries as a reliable data collection instrument • Gershuny (2000) • compares diary information with retrospective questionnaire: memory effect, loss of information • What type of data? Lots of information, spread in time : • itineraries (National Transportation Survey) • spending (Household Budget Survey) • daily activities (Time Use Survey) The point as regards quality is: prevent loss of information due to memory = must be filled BY RESPONDENT, all over a time period - day(s), week(s), month(s)…

  3. INTRODUCTION Issues • Diary = quality IF • Filled over time, not in retrospect • Doesn’t amplify non-response • Compliance? • Problem: a written medium + respondents left alone, autonomous. Difficult to control the data collection process as no one is present. • What can we learn from previous surveys?

  4. OUTLINE • The data: 2 INSEE surveys • Diary non-response: household level • Diary non-response: individual level • Diary quality: when and by whom are diaries actually filled?

  5. 1. Two diary-based surveys 1.1. The 1998 TUS • Time Use Survey • February 1998 to February 1999 • 12 000 addresses in the sample • 1 diary for each household member  15 • what is recorded = ACTIVITIES • 10-minute time slots • « natural language ». Ex: « washed the dishes »

  6. 1. Two diary-based surveys The TUS diary: left page

  7. 1. Two diary-based surveys The TUS diary: left page - example

  8. 1. Two diary-based surveys The TUS diary: right page - example

  9. 1. Two diary-based surveys 1.2. The 2006 HBS • Household Budget Survey • March 2005 to March 2006 • 20 000 addresses in sample • 2 diaries for each household member  15 • 2 consecutive weeks • What is recorded: purchases • Receipts can be stapled in the diary but need « commenting » • Purchases without receipt must be written down by the household

  10. 1. Two diary-based surveys The HBS diary

  11. 1. Two diary-based surveys The HBS diary - example

  12. 2. Diary non-response: household level • Several steps when dropout can occur: • HH refuses to answer the survey from the start • HH doesn’t complete Interview n°1 • HH reaches the end of ITW n°1 but refuses all diaries • ≥ 1 diary is accepted but no ITW n°2 can be done • ITW n°2 is done but no diary is filled when the interviewer returns, and HH won’t fill them with the interviewer

  13. 2. Diary non-response: household level 2.1. Non-response figures

  14. 2. Diary non-response: household level 2.2. Methodology • Simple logistic regression : refusing the diary=1 • Explanatory variables: only what is collected at the beginning of ITW1 to have information on a maximal number of households, even dropouts 3 different definitions of the outcome, for HBS: • Refusing to take even 1 diary • Eschewing ITW2 • Not having filled a single diary when the interviewer returns Then 2) + 3), and 1) + 2) + 3) together. Is it the same thing? Same type of HH in 3 behaviors?

  15. 2. Diary non-response: household level 2.2. Methodology • Explanatory variables: • Occupational category of HH head • Education level • Type of household (couple or not, with or without children) • Age of HH head • Region of residence • Survey Wave ( = time of the year) • Number of employed persons in the HH • Subsidized housing • Rural / urban + size of the urban area • + for TUS: HH income (collected at the beginning of survey) • Several specifications were used: with / without income; OLS / logistic. Same results.

  16. 2. Diary non-response: household level 2.3. Results on refusing to take diary

  17. 2. Diary non-response: household level 2.3. Results: HBS details Is there a difference between refusing a diary and not giving it back?

  18. 2. Diary non-response: household level Conclusion on households with no diary • Elderly HH are much harder to convince • The self-employed are more reluctant, esp. farmers for TUS and business owners for HBS • Regional (dis)inclination towards NIS surveys • Refusing to take a diary is different from not doing ITW2. • No ITW2 = refusal (>75) + lack of availability • Vacations • Household w/out couple

  19. 3. Diary non-response: individual level 3.1. TUS What do we call « individual diary non response »? Among the 8186 HH who reached the end of the 1st ITW (Little information was collected on the data collection process itself during the 1998 TUS)

  20. 3. Diary non-response: individual level 3.2. A specificity of the HBS What do we call « individual diary non response »? • Some individuals can write down their purchases on another household member’s diary: a kind of proxy (the relevant info is at the household level). • Mostly men and teenagers • They are considered as having filled their diary, even if we might think the quality is not as good as if they had had their own (loss of information caused by proxy)

  21. 3. Diary non-response: individual level 3.2. HBS: more detailed information What do we call « individual diary non response »? Among the 10240 HH for whom we have the individual variables + Is there a difference between the 2 categories of non-respondents?

  22. 3. Diary non-response: individual level 3.2. Refusal or avoidance? Is there a difference between the 2 categories of non-respondents? Assuming people with no diary did not want to fill it, who said so and who only didn’t give it back? • Refuse from the start to take a diary (N=224) • More skilled • 15 to 25 • Parisians • Above 55 and even more, above 75 • Do not give it back (N=363) • Self-employed • Less skilled

  23. 3. Diary non-response: individual level 3.3 No diary: explanatory variables Explanatory variables: • Gender • Age • Lives in a couple • Education • Nationality (French by birth, became French, EU, non-EU) • Occupation (6 categories + Student + Retired + Other Non LF participant) • Region of residence

  24. 3. Diary non-response: individual level 3.4. No diary : results Are the 2 surveys’ diary non-response determined by the same factors? Dependent variable : no diary = 1

  25. 4 items, answered by the interviewer Away: 21.5% Unable to (handicapped, illiterate): 12% Refusal: 42% Other reasons: 10.5% N/A: 14% Are more prone to refusal 35 to 55 (60%) No diploma (37%) Are away 15 to 25 (45%) Men Are unable to fill the diary No diploma (31%) Citizenship has little, if any influence Variable on handicap not available 3. Diary non-response: individual level 3.4. Cause of missing diary: TUS

  26. 5 items, answered by the interviewer at the end of ITW1: Away: 58% Unable to (handicapped, illiterate): 18% Refusal: 12% Other reasons:12% Are more prone to refusal Men (15%) Self-employed (25%) 45 to 55 (40%) Near zero refusal among college graduates Are away 15 to 35 College graduates Are unable to fill it No diploma (43%) Handicapped (60%) Elderly >75 (68%) 3. Diary non-response: individual level 3.4. Cause of missing diary: HBS Excluding the 30% of individuals who do not receive a diary because they will report on someone else’s

  27. 3. Diary non-response: individual level Conclusion on individuals lacking a diary • Very similar explanatory variables in the 2 surveys (except regions) • Effect of educational level more important for TUS • easily understandable since it is much more of a narrative exercise • The possibility of a “proxy” (reporting on someone else’s diary) avoids losing too many men in the HBS • Teenagers living with their parents are very difficult to catch

  28. 4. Indicators of diary quality 4.1. When is the diary filled? HBS TUS

  29. 4. Indicators of diary quality 4.1. When is the diary filled? • Women are better compliers • HBS: 85% every day VS 80% of men • TUS: 38% all along the day VS 31% of men • The older the respondent, the later the diary is filled • HBS: • TUS:

  30. 4. Indicators of diary quality 4.1. When is the diary filled? • Occupation: clear differences • Respondents not in the LF (housewives, retirees) fill the TUS diary more often “all along the day”, as requested. • Farmers and other self-employed fill the diary later: “in the evening without notes”, “the day after” (TUS) or “just before or during ITW2” (HBS) • Tertiary sector wage earners (employees, managers, “intermediate occupations”) typically fill the diary “in the evening”, with or without notes (TUS), or every day (HBS)

  31. 4. Indicators of diary quality 4.1. When is the diary filled? • Education: no clear pattern except for those w/out diploma • HBS: 76% of “no education” filled diary everyday, 15% irregularly, 9% just before or during ITW2 • TUS: 30% all along the day, 22% “other” • Citizenship: language barrier • HBS: 10% just before or during ITW2 • TUS: 20% “other” for non-French citizens Again: TUS = more difficulties

  32. 4. Indicators of diary quality 4.1. When is the diary filled? • Region: Constants and variations • TUS: West, East and ‘Center-East’ comply better (+ 4pts), Mediterranean much less (-7pts) • HBS: East, West and ‘Center’ comply better (+ 2 to + 4 pts), South-West less, Mediterranean = OK

  33. 4. Indicators of diary quality 4.2. By whom is the diary filled? (HBS only)

  34. The interviewer fills the diary more often when the respondent is: Living alone (6.6%) Without degree (9%) Handicapped (10%) A farmer (12%) Elderly (15%) 4. Indicators of diary quality 4.2. By whom is the diary filled? (HBS only)

  35. Someone else in the HH fills the diary (not incl. those with a « proxy diary ») more often when the respondent is : Without degree or less than high school (8%) Living in the North of France (10%) Self-employed or industrial worker (11%, VS. 4% of white-collar workers, mainly women) A man (11%, VS. 3% of women) Under 25 (30%) or between 25-35 (10%) 4. Indicators of diary quality 4.2. By whom is the diary filled? (HBS only)

  36. Quality is less good at the ends of age distribution: Youth living with their parents Elderly Men escape data reporting much more than women: We know less on how they spend their money and time Self-employed are more difficult than wage-earners: Same as other surveys, they are a less well-known population Regional variations have some constants (East > Paris and South-East) as well as much unexplained variability CONCLUSION

  37. Much happens during the second ITW A lot of information is « rescued » Elderly Less skilled Handicapped Non-native speakers Importance of the interviewer as a mediator when the information must be written down Survey design must take that into account and allow interviewer to spend time on those respondents CONCLUSION

More Related