1 / 58

Imputation in the 2011 Census

Imputation in the 2011 Census. NILS Brownbag Talk – 6 May 2014 Richard Elliott. Overview. Background What is imputation How did we impute the 2011 Census Strategy Process Implementation Considerations Information Next steps. Background.

oona
Download Presentation

Imputation in the 2011 Census

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Imputation in the 2011 Census NILS Brownbag Talk – 6 May 2014 Richard Elliott

  2. Overview • Background • What is imputation • How did we impute the 2011 Census • Strategy • Process • Implementation • Considerations • Information • Next steps

  3. Background • Legal obligation on the public to complete a Census Questionnaire accurately • A minority didn’t provide such information • Item non-response • Leave questions unanswered • Make mistakes (i.e. neglect to follow questionnaire instructions) • Provide values that are out of range (e.g. Born in 1791) • Item inconsistency • Captured values not consistent with other values on the questionnaire (e.g. 6 year old mother) • Non-response • Don’t fill in the questionnaire at all

  4. Background • It is NISRA’s policy to report estimates for the entire population. Therefore Imputation was utilised to: • Correct for non-response • Estimate the missing persons and households • Correct for Item non-response • Fill the gaps left by unanswered questions • Correct for Item inconsistency • Ensure that the information provided is consistent • These types of data quality issues apply equally to any data collection exercise • Not specific to Census • Census Office recognises that users need to be aware

  5. Background • While imputation was used to “fill the gaps”, its strength comes from the information that was recorded • Therefore, it is important to recognise the following: • Responses to the Census represent a self-assessment of a respondents circumstances • Proxy responses given by main householder • Respondents didn’t always complete the questionnaire correctly • 85% of questionnaires were completed on paper forms • handwriting that had to be captured using an electronic character recognition system • While Service Levels in place for capture, errors still possible

  6. Two Types of Imputation • Item Edit and Imputation • Correcting a dataset for inconsistencies and item non-response • Making each record “complete and consistent” • Record imputation • The addition of whole records to a dataset • Estimate and adjust for persons missed, duplicated and counted in the wrong place • Increases the accuracy of the overall estimates

  7. Item Edit and Imputation Strategy • Primary Objective: • to produce a complete and consistent database where unobserved distributions were estimated accurately by the imputation process • There were three key principles • All changes made maintain the quality of the data • The number of changes to inconsistent data are minimised • As far as possible, missing data should be imputed for all variables to provide a complete and consistent database

  8. Item Edit and Imputation Strategy • In adhering to these principles, the following key aims were defined • Editing must not introduce bias or distortion in the data • Editing facilitates the production of output data that is fit for purpose • Editing methods help to ensure that pre-determined levels of data quality are met • Highest priority given to variables which define population bases (e.g. Age and Sex) • Editing supports the production of the population estimates by ensuring that the basic population estimates are accurate

  9. Item Edit and Imputation Strategy • Used a similar but enhanced version of the framework adopted in 2001 • One Number Census Process • Tried and tested in 2001 • Was undertaken as part of the Downstream Processing (DSP) project at ONS • Included both Item and Record Imputation • Supplemented by detailed QA at every stage by NISRA Census Office • NISRA benefitted from enhancements to the system found through ONS data processing • Ultimately NISRA responsible for processing of NI data and any parameter tweaking / re-runs

  10. Imputation Process – 4 Key Stages 1. Cleansing the Data 2. Item Imputation

  11. Imputation Process – 4 Key Stages 1. Cleansing the Data 2. Item Imputation

  12. Implementation – Capture and Coding • Capture and coding rules • Turning tick and text responses into data that could be edited and imputed • Complex coding used to assign numerical values to written text and ticked boxes (e.g. occupation and industry coding) • Invalid responses flagged for imputation (V, W, Y and Z) • Determinations made to responses to resolve combinations of tick and text • Ticks that could not be determined set to W (failed multi-tick) • Text that was uncodeable set to V (uncodeable text response) • Data subject to checks to ensure each question response was within a predefined range (e.g. No year of birth before 1896 or after 2011) • Invalid responses set to Z (out of range) • Missing data flagged as Y (missing requires imputation)

  13. Implementation – Capture and Coding • Determining combinations of ticks • Single tick 

  14. Implementation – Capture and Coding • Determining combinations of ticks • Resolvable multi-tick  

  15. Implementation – Capture and Coding • Determining combinations of ticks • Irresolvable multi- tick    This will be assumed missing and imputed.

  16. Implementation – Capture and Coding • Missing data This will be imputed.

  17. Implementation – Capture and Coding • Resolving write-ins • Numbers 1 01

  18. Implementation – Capture and Coding • Resolving write-ins • Numbers two 02

  19. Implementation – Capture and Coding • Resolving write-ins • Range Check 199 This will be assumed missing and imputed.

  20. Implementation – Capture and Coding • Resolving write-ins • Codeable response “FRANCE” gets coded to 250  F R A NC E

  21. Implementation – Capture and Coding • Resolving write-ins • Uncodeable response “SUGAR” is clearly not a country so set to set to VVV. This will be assumed missing and imputed.  S U G A R

  22. Imputation Process – 4 Key Stages 1. Cleansing the Data 2. Item Imputation

  23. Implementation – RMR • Reconcile Multiple Responses (RMR) • Removal of false persons • Removal of persons generated by capture anomalies • For example: strike throughs, inadequately completed questionnaires • Removal of duplicates (multiple persons / households) • Individuals who included themselves more than once • Separated parents who included their children at both addresses • Creating households / communals from multiple questionnaires • Consolidating H4 / HC4 / I4 etc • Validation • Renumbering person records within households / communals

  24. Imputation Process – 4 Key Stages 1. Cleansing the Data 2. Item Imputation

  25. Implementation – FRDVP • Filter Rules and Derived Variables for Processing (FRDVP) • Correct data by applying edits to correct for questionnaire routing errors • Apply hard edits to keep individual records consistent • Minimal at this stage (mostly applied within imputation system) • Information not required set to X • No imputation done on any variable set to X • Create high level variables to be used within the Item Imputation system • Blocking variables for donor searching • Makes it easier to find donors

  26. Implementation – FRDVP • Surplus information – questionnaire routing In this scenario the respondent should have skipped question 6 and gone straight to question 7. Therefore, since the respondent should not have answered question 6, it is set to: X (not required)   

  27. Implementation – FRDVP • Surplus information – questionnaire consistency 0 1 0 1 2 0 1 1  9 L O RD WAR DE N S In this scenario, since the respondent is aged under 1 on Census day, and therefore did not have a usual address one year ago, the captured address information is set to X. C RE S C E N T B T 1 9 1 Y J

  28. Imputation Process – 4 Key Stages 1. Cleansing the Data 2. Item Imputation

  29. Implementation – Item Imputation • Achieved using CANCEIS • Canadian Census Edit and Imputation System • Developed specifically for Census type data • ie a mix of categorical and numerical variables • Donor-based edit and imputation system that can simultaneously: • Apply nearest-neighbour donor imputation • Apply deterministic edits and maintain consistency • Evaluated and endorsed as the 2011 Census imputation tool • Faster • Less resource intensive • Allowed for more joint-imputation

  30. Implementation – CANCEIS • How did CANCEIS work in practice • The database was divided up into processing units for the purposes of resource management and maximising donor pools Household questions Individual imputation Donor unit = household Three Geographic units Household Persons 1 to 6 Joint Household imputation Between person edits and relationships Donor unit = household of same size Household Persons 7 to 30 Individual imputation Relationship to Person 1 Donor unit = individual person Person questions Communal Persons Individual imputation Donor unit = individual person

  31. Delivery Groups (Processing Units)

  32. Implementation – CANCEIS • How did CANCEIS work in practice • The database was divided up into processing units for the purposes of resource management and maximising donor pools Household questions Individual imputation Donor unit = household Three Geographic units Household Persons 1 to 6 Joint Household imputation Between person edits and relationships Donor unit = household of same size Household Persons 7 to 30 Individual imputation Relationship to Person 1 Donor unit = individual person Person questions Communal Persons Individual imputation Donor unit = individual person

  33. Implementation – CANCEIS • How did CANCEIS work in practice • The household questions were imputed within a single module • Person data was divided up into 4 modules • Aim was to group variables that help predict each other • Attempt to maximise the number of donors for a given group Demographics e.g. Age, Sex, Marital status, Student, Activity last week Culture e.g. Ethnicity, Country of birth, Language, Passports Health e.g. General health, Disability, Long-term condition Labour Market e.g. Economic activity, Hours worked, Qualifications

  34. Implementation – CANCEIS • How were the donors selected? • Within each module a number of matching variables were used to select donors • Matching variables were weighted according to several factors • How well they would predict other values and how highly they should be prioritised when resolving inconsistencies • For example, age is often a good predictor of other demographic variables • Age was given a high weight, therefore observed ages were prioritised over other values if there was an inconsistency and changes were required • Northings and Eastings were used to control for geographical differences and find donors from similar areas • These were given a small weight compared to demographic characteristics like age, sex and marital status etc

  35. Implementation – CANCEIS • Matching variables (example) • Suppose someone omitted to fill in their occupation details • The record would be flagged for imputation under the Labour Market module • Donor pool identified by matching on (for example): • Economic Activity • Industry • Hours worked • Qualifications • These variables deemed to influence Occupation • Occupation information imputed from a donor with similar Labour Market characteristics

  36. Implementation – CANCEIS • Editing and Imputing was done simultaneously • Each record was checked for consistency before imputation • Any items that failed the checks were marked for imputation along with the missing items • A single donor was selected to resolve inconsistencies and non-response • Only values which satisfied the edit constraints were imputed into the recipient record • CANCEIS sought to minimise the number of changes required to repair a record when edit constraints were in place • There were 31 edit rules which were broadly based on 2001 • e.g. If aged between 5 and 15 then must be in full-time education • Some rules had to be updated to account for changes since 2001 • e.g. Removal of rule that did not allow same-sex couples • Replaced with rules that said married couples had to be opposite-sex and civil partners had to be same-sex

  37. Implementation – CANCEIS • Say we have the following (oversimplified) example: • Student is missing • Requires imputation under the demographic module • This record is subject to two edit constraints • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Fails Rule A since aged 10 and married • Therefore, both Age and Marital Status are also flagged for imputation

  38. Implementation – CANCEIS • Say we have the following (oversimplified) example: • Student is missing • Requires imputation under the demographic module • This record is subject to two edit constraints • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Fails Rule A since aged 10 and married • Therefore, both Age and Marital Status are also flagged for imputation

  39. Implementation – CANCEIS • The system searches for potential donors • Matching on demographic variables • Uses Northings and Eastings to find a donor in the area • The following records are returned:

  40. Implementation – CANCEIS • Must be aged 16+ to be married • Aged 5 to 15 must be a student

  41. Implementation – CANCEIS • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Donor1 • Using Donor1 would mean that “Single” is taken as well as “No”

  42. Implementation – CANCEIS • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Donor1 • Using Donor1 would mean that “Single” is taken as well as “No” • The new record fails Rule B • Therefore Age is taken from the donor as well

  43. Implementation – CANCEIS • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Donor1 • Using Donor1 would mean that “Single” is taken as well as “No” • The new record fails Rule B • Therefore Age is taken from the donor as well Two observed value changes

  44. Implementation – CANCEIS • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Donor2 • Using Donor2 would mean that “Single” is taken as well as “Yes”

  45. Implementation – CANCEIS • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Donor2 • Using Donor2 would mean that “Single” is taken as well as “Yes” • The new record passes both Rule A and Rule B Only one observed value change

  46. Implementation – CANCEIS • Must be aged 16+ to be married • Aged 5 to 15 must be a student • Donor2 • Using Donor2 would mean that “Single” is taken as well as “Yes” • The new record passes both Rule A and Rule B • Donor2 given a higher probability of selection Only one observed value change

  47. Implementation – CANCEIS • Points to note • Variables were imputed in blocks of similar variables (modules) • there was no individual model for any one question • There is independency between the modules • for example, cultural characteristics might come from a different donor to employment characteristics • Imputed person data was combined in a way that maintained relationship consistency within a household • Given the processing approach, quality was maintained at the geographic unit level

  48. Imputation Process – 4 Key Stages 1. Cleansing the Data 2. Item Imputation

  49. Implementation – Manual Imputation • Manual Imputation kept to a minimum but was necessary • Manual Imputation – QA checks • Quality Assurance at every stage of processing • Distributional checks and checks against comparator data sources • Edits made through Data File Amendments (DFAs) • DFAs not taken lightly • Involved detailed questionnaire image analysis • Mostly correcting for capture errors • e.g. Centenarians • Manual Imputation to increase donor pool • Temporary changes sometimes required when donor pool too small • E.g. Postcode matching (would have been done later in processing but brought forward)

  50. Imputation Process – 4 Key Stages 1. Cleansing the Data 2. Item Imputation

More Related