270 likes | 480 Views
Census Editing and the Art of Motorcycle Maintenance. Michael J. Levin Center for Population and Development Studies Harvard University Michael.levin@yahoo.com. The Census Process. Data collection Capture Editing Tabulation and Dissemination Archiving. History of census editing.
E N D
Census Editing and the Art of Motorcycle Maintenance Michael J. Levin Center for Population and Development Studies Harvard University Michael.levin@yahoo.com Harvard Center for Population and Development Studies
The Census Process • Data collection • Capture • Editing • Tabulation and Dissemination • Archiving Harvard Center for Population and Development Studies
History of census editing • Early years – manual or nothing • Computers • Within record editing • Between record editing • Hot decking Harvard Center for Population and Development Studies
What is editing • Editing is the systematic inspection of invalid and inconsistent responses, and subsequent manual or aurtomatic correction according to pre-determined rules. • The editing team!! Harvard Center for Population and Development Studies
Why edit? • Edited vs unedited data • Always preserve original data • Consider the users!! Harvard Center for Population and Development Studies
Table 1. Sample population by 15-year age group and sex, using unedited and edited data Harvard Center for Population and Development Studies
Initial data sets contain errors • How over-editing is harmful • Timeliness • Finances • Distortion of true values • False sese of security Harvard Center for Population and Development Studies
What we have to look out for • Treatment of unknowns • Spurious changes • Using tolerances • Learning from the editing process • Quality assurance • Costs of editing Harvard Center for Population and Development Studies
Types of Correction • Manual correction • Names • Sex • Automatic correction • Assign an unknown • Assign a value • Impute a value Harvard Center for Population and Development Studies
Types of editing • Top Down • The usual way • Is simple and straight forward • Multiple-variable editing approach • Uses more information • Is likely to be a better guess Harvard Center for Population and Development Studies
Two parts of a national edit • Structure editing • Content editing Harvard Center for Population and Development Studies
Methods of Correction and Imputation • When imputation is not needed – toggling sexes • Static imputation – cold deck technique • Dynamic imputation – hot deck technique Harvard Center for Population and Development Studies
Goals of the edit • Imputed household should closely resemble failed edit household • Imputed data should come from a single donor person or house resembling donee • Equally good donors should have equal chances Harvard Center for Population and Development Studies
Figure 1. Sample editing specifications to correct sex variable, in pseudocode Harvard Center for Population and Development Studies
Hot Deck • Geographic considerations • Use of related items • Order of the items changes the matrices • Complexity of the imputation matrices Harvard Center for Population and Development Studies
In developing hot decks • Imputation matrices – structure of the matrices • Standardized imputation matrices • Seeding the decks • Big, but not too big • Understanding what the matrix is doing • When the matrix is too small … • Occupation and industry!! Harvard Center for Population and Development Studies
Aids to checking edits • Listings • Writing whole households before and after with changes • Frequency matrices Harvard Center for Population and Development Studies
Figure 4. Example of a listing summary for Malawi 2008 Census[LISTING] Harvard Center for Population and Development Studies
Figure 5. Example of a listing summary for Lesotho 2006 Census[LISTING] Harvard Center for Population and Development Studies
Figure 8. Example of a write listing for Ethiopia 2007 Census[WRITE] Harvard Center for Population and Development Studies
Figure 10. Example of a frequency distribution for Sudan 2008 Census[FREQUENCY] Harvard Center for Population and Development Studies
Figure 11. Example of a frequency distribution for additional edit for Zambia 1990 Census[FREQUENCY] Harvard Center for Population and Development Studies
Other considerations • Running the edit three times: seed, run, check • Saving original responses • Imputation flags Harvard Center for Population and Development Studies
Conclusions • Edits part of the series of census procedures • Usually more for aesthetics than technical enhancement • Hardware and software changing rapidly • The revolution continues! Harvard Center for Population and Development Studies