Work Session on Statistical Data Editing Ljubljana, Slovenia

1. The Evolution of Edits in the Canadian Census of Population Online Questionnaires Work Session on Statistical Data Editing Ljubljana, Slovenia Danielle Laroche May 9-11, 2011

2. Overview of the Canadian Census What we have learned about Internet data collection over the past 10 years The new wave methodology Design and characteristics of the online options Types of edits Future plans Presentation Outline Statistics Canada • Statistique Canada

3. Next Census: May 10, 2011 Every five years Mandatory Mainly self-enumeration Four response channels: Paper questionnaire Internet Census Help Line Non-response Follow-up (NRFU) 12.4 M dwellings / 31.6 M persons (2006) Canadian Census overview Statistics Canada • Statistique Canada

4. Internet option has many benefits For both Statistics Canada and respondents Data quality is the most important aspect Rejection rates: - Short Form: Paper = 5.6% vs. Internet = 2.5% (2006) - Long Form: Paper = 39.1% vs. Internet = 5.7% Most important problems were mainly due to the technology (drop-down menus) Edit messages impact respondents’ behaviour: (+) Less item non-response, fewer invalid responses (±) False perception that a question must be answered to continue (–) When an answer is unknown, messages might encourage valid but incorrect answers What we have learned over the past10 years Statistics Canada • Statistique Canada

5. Internet use is progressing 2001 = 5%, 2004 = 7%, 2006 = 18.3%, 2009 = 41% But still far from its potential Internet home penetration rate¹: 2000 = 40%, 2004 = 58%, 2006 = 70% Front cover of questionnaires is not a good Internet promotion tool Sending a questionnaire by mail doesn’t encourage Internet response Sending a letter instead of a questionnaire is very effective for increasing Internet response, but there is also the potential to increase the NRFU (tested 2006) ¹ Household Internet Use Survey (STC) What we have learned over the past10 years (cont’d) Statistics Canada • Statistique Canada

6. New methodology(Tested in 2009) Internet response much higher with a letter Letter 41% vs. Paper 14.8% Voice broadcast phone message is a cost-effective follow-up method Phone numbers on census frame linked to dwellings that respond more NRFU workload was initially higher but was similar to the benchmark by the end What we have learned over the last 10 years(cont’d) Statistics Canada • Statistique Canada

7. 2011 Census wave methodology • CD = Census Day Statistics Canada • Statistique Canada

8. Major modifications(summer 2010) Abolition of the mandatory long form questionnaire National Household Survey (NHS) - voluntary: 65 questions (similar to former long form) Target 30% of dwellings (4.5 M) Census- mandatory: 10 questions (former short form plus two questions on language) Target 100% of dwellings Modifications to the 2011 Census Statistics Canada • Statistique Canada

9. 2011 Census and National Household Survey Census wave methodology Expect 40% Internet response National Household Survey Integrated to the Census / Same time frame Wave approach similar to the Census but depends on Census response status: Census Internet respondents Other Census respondents Census non-respondents NRFU on a sub-sample of non-respondents Statistics Canada • Statistique Canada 2019-12-20 9

10. Questions and answer choices: Identical to paper versions to facilitate integration of data and to minimize potential mode effects But at the same time… Take advantage of the technology to improve data quality Radio buttons, checkboxes, drop-down menus, automated flow, online help, save and finish later, progress bar, text resizing Mostly Matrix format Sequential format use for questions on labour and income In the 2006 Census, edits were performed one question at a time Design and characteristics of the online options Statistics Canada • Statistique Canada

11. Common Look and Feel (CLF) - 2.0 into effect in 2007 Standards and guidelines for presenting content in Canadian federal government websites Ensure functionality of the online questionnaires with or without JavaScript Used to perform edits while the respondent is entering data (in 2006, JavaScript had to be enabled) Major impacts on the functionality and design of 2011 online questionnaires New edits and messages were developed Modifications made to the 2006 online questionnaires Statistics Canada • Statistique Canada

12. Mandatory or hard edits (provide messages) Number of persons and names Confirmation pages (provide instructions) Age (validate date of birth) and income (validate 14 sources) Soft edits (provide messages) Non-response Partial response Invalid response New edits for 2011 JavaScript (interactive edits) Types of edits Statistics Canada • Statistique Canada

13. Mandatory or hard edit - (1)Number of persons Message, question and field to be corrected are in red Statistics Canada • Statistique Canada

14. Mandatory or hard edit (2) - Names Non-response (no name) for P3 2019-12-20 14 Statistics Canada • Statistique Canada

15. Confirmation page (1) - Age Instruction to modify answers On previous page: P1 = valid date of birth P2 = invalid date of birth P3 = non-response Statistics Canada • Statistique Canada

16. Confirmation page (2) – Summary of income (NHS) Instruction to modify answers Statistics Canada • Statistique Canada

17. The most frequent edit (almost each question) Effective in obtaining answers to questions that respondents might have overlooked However, messages lead respondents to believe that they must provide an answer to continue This might impact data quality For example, some respondents reported that they lived in a different city 5 years ago but entered their current postal code Non-response edit and messages for all PC were disabled Non-response edits Statistics Canada • Statistique Canada

18. Used in 2006 Census for questions with “other - specify” answers When users entered a write-in, the corresponding radio button or checkbox was automatically selected (JavaScript) The new CLF introduces the possibility of inconsistent responses A write-in response might now have an inappropriate radio button selected E.g., Question 49b – Other languages at work Partial response edits Statistics Canada • Statistique Canada

19. Q49b – Other languages at work (without JavaScript) Partial response edits (cont’d) The corresponding checkbox is not marked A write-in is provided Statistics Canada • Statistique Canada

20. In the 2006 Census, invalid response edits were used for postal code and numeric fields Postal code not in format ANA NAN - ex., K1A 0T6 For numeric fields, a message appeared if the answer was outside the range established for a question Mode effect: Number of hours worked last week The ‘0’ was changed for “1” hour after receiving the valid range message (between 1 and 168) instead of selecting “None” below All numeric fields were modified to accept a “0” value Again, the new CLF introduces the possibility of invalid or multiple responses E.g., Yes and No responses should not be selected together Invalid response edits Statistics Canada • Statistique Canada

21. New edits were developed to assess their feasibility and the acceptability of more complex edits Consistency between two questions Year of birth and year of immigration Looking for a more precise response Answers such as “construction” or “engineer” to questions on industry and occupation Avoiding double reporting of amounts Double counting between parents for child care expenses New edits for 2011 Statistics Canada • Statistique Canada

22. Evaluation of 2011 online questionnaire edits Error Pattern File (up to four messages recorded per person or question) Statistics Canada E-services initiative E-File transfer service E-Questionnaire service Primary collection mode 130 surveys, including the 2016 Census Future Plans Statistics Canada • Statistique Canada

23. For more information, Pour plus d’information, please contact: veuillez contacter : Danielle Laroche danielle.laroche@statcan.gc.ca Thank you Statistics Canada • Statistique Canada