Census Editing and the Art of Motorcycle Maintenance

1. Census Editing and the Art of Motorcycle Maintenance Michael J. Levin Center for Population and Development Studies Harvard University Michael.levin@yahoo.com Harvard Center for Population and Development Studies

2. Harvard Center for Population and Development Studies

3. The Census Process • Data collection • Capture • Editing • Tabulation and Dissemination • Archiving Harvard Center for Population and Development Studies

4. History of census editing • Early years – manual or nothing • Computers • Within record editing • Between record editing • Hot decking Harvard Center for Population and Development Studies

5. What is editing • Editing is the systematic inspection of invalid and inconsistent responses, and subsequent manual or aurtomatic correction according to pre-determined rules. • The editing team!! Harvard Center for Population and Development Studies

6. Why edit? • Edited vs unedited data • Always preserve original data • Consider the users!! Harvard Center for Population and Development Studies

7. Table 1. Sample population by 15-year age group and sex, using unedited and edited data Harvard Center for Population and Development Studies

8. Initial data sets contain errors • How over-editing is harmful • Timeliness • Finances • Distortion of true values • False sese of security Harvard Center for Population and Development Studies

9. What we have to look out for • Treatment of unknowns • Spurious changes • Using tolerances • Learning from the editing process • Quality assurance • Costs of editing Harvard Center for Population and Development Studies

10. Types of Correction • Manual correction • Names • Sex • Automatic correction • Assign an unknown • Assign a value • Impute a value Harvard Center for Population and Development Studies

11. Types of editing • Top Down • The usual way • Is simple and straight forward • Multiple-variable editing approach • Uses more information • Is likely to be a better guess Harvard Center for Population and Development Studies

12. Two parts of a national edit • Structure editing • Content editing Harvard Center for Population and Development Studies

13. Methods of Correction and Imputation • When imputation is not needed – toggling sexes • Static imputation – cold deck technique • Dynamic imputation – hot deck technique Harvard Center for Population and Development Studies

14. Goals of the edit • Imputed household should closely resemble failed edit household • Imputed data should come from a single donor person or house resembling donee • Equally good donors should have equal chances Harvard Center for Population and Development Studies

15. Figure 1. Sample editing specifications to correct sex variable, in pseudocode Harvard Center for Population and Development Studies

16. Hot Deck • Geographic considerations • Use of related items • Order of the items changes the matrices • Complexity of the imputation matrices Harvard Center for Population and Development Studies

17. In developing hot decks • Imputation matrices – structure of the matrices • Standardized imputation matrices • Seeding the decks • Big, but not too big • Understanding what the matrix is doing • When the matrix is too small … • Occupation and industry!! Harvard Center for Population and Development Studies

18. Aids to checking edits • Listings • Writing whole households before and after with changes • Frequency matrices Harvard Center for Population and Development Studies

19. Figure 4. Example of a listing summary for Malawi 2008 Census[LISTING] Harvard Center for Population and Development Studies

20. Figure 5. Example of a listing summary for Lesotho 2006 Census[LISTING] Harvard Center for Population and Development Studies

21. Figure 8. Example of a write listing for Ethiopia 2007 Census[WRITE] Harvard Center for Population and Development Studies

22. Figure 10. Example of a frequency distribution for Sudan 2008 Census[FREQUENCY] Harvard Center for Population and Development Studies

23. Figure 11. Example of a frequency distribution for additional edit for Zambia 1990 Census[FREQUENCY] Harvard Center for Population and Development Studies

24. Other considerations • Running the edit three times: seed, run, check • Saving original responses • Imputation flags Harvard Center for Population and Development Studies

25. Conclusions • Edits part of the series of census procedures • Usually more for aesthetics than technical enhancement • Hardware and software changing rapidly • The revolution continues! Harvard Center for Population and Development Studies