Overview of Census Evaluation and Selected Methods Pres. 2

1. Overview of Census Evaluationand Selected MethodsPres. 2

2. Why evaluate ? Because the census is a huge operation (size, number of persons involved) prone to errors To provide users with some measures of quality of census data to help them interpret the results To identify types and sources of error in order to assist the planning of future censuses To serve as a basis for constructing a best estimate of census aggregates, such as total population, or to provide adjustment ofcensus results But…not to criticize the census takers !!

3. What errors in a census? Coverage errors: Errors in the count of persons or housing units resulting from cases having been “missed” or counted erroneously Content errors: Errors in the recorded characteristics of persons or housing units enumerated in the census (e.g. wrong age...)

4. Coverage errors Omissions Missing housing units, households and/or persons during census enumeration If the whole housing unit is missed, all households and persons living in the housing unit will also be missed Major causes of omissions are: failure to cover whole land area of a country in creating EAs; Mistakes made by enumerators in canvassing assigned areas Ambiguous definitions of EAs, unclear boundaries of EAs, faulty maps or coverage error during the pre-census listing exercise

5. Coverage errors Omissions contd. In addition, omissions within EAs can result because all or some of the members of the household were not present at the time of enumeration Proxy respondents can inadvertently or deliberately omit some members of a household

6. Coverage errors Duplications Occur when persons, households or housing units are counted more than once Reasons for duplications include: Overlapping of enumerator’s assignments owing to errors done during pre-census listing and delineation Failure by enumerators to clearly identify boundaries In practice, the number of omissions usually exceeds the number of duplicates (net under-counts)

7. Coverage errors Erroneous Inclusions This includes: Housing units, households and persons enumerated while they should have not been (e.g. babies born after the census reference date) Or enumerated in a wrong place

8. Coverage errors Gross error This is the sum of duplications, erroneous inclusions and omissions Net error This is the difference between over-counts and under-counts Net census under-count exists when number of omissions exceeds the number of duplicates and erroneous enumerations Net census over-count is the converse

9. Methods for evaluation of census errors Single Source of Data Demographic analysis of the census Interpenetration studies Multiple Sources of Data Non-matching studies Demographic analysis using previous censuses Comparison with administrative sources or existing surveys Matching studies Post Enumeration Surveys Record checks

10. Single Source of Data Demographic Analysis of the Census Average number of persons per household Sex- and age- ratios Tabulations... For an overall assessment of quality: an age pyramid is a standard method stable population analysis can be undertaken as long as assumptions pertaining to constant fertility and mortality and no migration are met, for countries with declining mortality a quasi-stable model may be appropriate

11. Single Source of Data Interpenetrating studies Method involves drawing subsamples, selected in an identical manner, from the census frame Each subsample should be capable of providing valid estimates of population parameters Assignment of personnel (i.e. enumerators, coders, data entry staff, etc.) is done randomly The method helps to provide an appraisal of the quality of census information and procedures

12. Multiple Sources of Data – Non matching studies Demographic Analysis Results from a census may be compared with data from other demographic systems such as vital registration systems For example, the cohort component method of demographic analysis uses successive censuses including life-table survival rates age-specific rates age-specific fertility rates and estimates of international migration

13. Multiple Sources of Data – Non matching studies Demographic Analysis contd. Population can be projected forward to the reference date of the second census based on estimated levels of and age schedules of fertility, mortality The expected population is then compared to the enumerated population in the current census Yet another method is the comparison of age distributions of successive censuses

14. Multiple Sources of Data – Non matching studies Demographic Analysis contd. Also the cohort survival method which is a regression method can be used, thus, population counts by age from two censuses and deaths by age during the inter-censal period are used to estimate coverage rate

15. Multiple Sources of Data – Non matching studies Comparison with existing household surveys In theory any probability sample of households or persons can be used to evaluate coverage and content error in a census if: They have identical items with same concepts and definitions They are independent from the census Must have been conducted close to the census date There should be sufficient identification information to facilitate accurate matching

16. Multiple Sources of Data – Matching studies Record checks Census records are matched with a sample of records from identification systems such as the vital registration system The relevant respondents to the census questionnaire are traced to the time synchronized with the census Sources include: Previous census Birth registrations School enrolment Citizen registration card Immigration registers etc.

17. Multiple Sources of Data – Matching studies Record checks contd. Both coverage and content errors could be measured through the above comparisons To evaluate coverage efficiently the following preconditions are essential: A large proportion of census population should be covered in record system The census and record system should be independent from each other There should be sufficient information in records

18. Multiple Sources of Data – Matching studies To evaluate content efficiently the following preconditions are essential: The record system should contain some relevant items covered in the census such as age, sex, education, relationship, marital status etc. Definitions of items should be identical between the census and the record system Countries that have used record checks include: Demark, Finland, Norway, Sweden, Taiwan and Canada

19. Multiple Sources of Data – Matching studies Post Enumeration Survey (PES) Consists of two separate coverage studies : A survey conducted using a sample frame independent of the census. Persons from this survey are matched to the census to estimate the number of persons missed in the census A survey conducted using a sample drawn from persons enumerated in the census. This sample is re-enumerated to determine if the sample person or unit was erroneously enumerated (inc. erroneously located)

20. Multiple Sources of Data – Matching studies Post Enumeration Survey (PES) contd. Results can be used to evaluate the reliability of some characteristics such as sex, age, marital status, relationship to reference person or head of the household. For some countries the results of PES can be used to adjust some census results Facilitates better interpretation of census results More discussion of PES is the focus of this workshop

21. Strengths and weaknesses of evaluation methods Single source: Methods that depend on a single datasource provide less insight into the magnitude and types of errors in the census data The merit is that the methods using such sources do not require additional data to be collected No need for sophisticated matching although this is also a limitation They provide a general impression of quality of the census data

22. Strengths and weaknesses of evaluation methods (contd.) Single Source -Interpenetrating studies Gives good idea of different contribution of component errors to total census error Helps to identify operational stages that contribute to census error, thus identifying procedural limitations in a census Demerits include: That it is an expensive operation demanding many field staff, intensive training and close supervision Relatively complex in designing and implementation

23. Strengths and weaknesses of evaluation methods (contd.) Multiple sources - Non-matching studies: Review census results at aggregate rather than unit level i.e. provides only estimates of net census error Evaluates very limited characteristics such as sex and age distributions Merit They are relatively cheap compared to matching studies

24. Strengths and weaknesses of evaluation methods (contd.) Non matching methods - Demographic Analysis: Advantage – no additional data is needed to be collected to perform the analysis Less costly In statistical offices with sufficient numbers of demographers there is no need for additional staff to do the technical analysis On the negative side these methods provide less insight into the different contributions of component errors to total error in the census Quality of sources (Vital Statistics…)

25. Strengths and weaknesses of evaluation methods (contd.) Matching methods: It provide separate estimates of coverage and content error Prospects of evaluating more characteristics compared to what can be done with non-matching studies Challenges Calls for high level technical skills including managerial Matching is expensive

26. Strengths and weaknesses of evaluation methods (contd.) Post enumeration survey Merits: Its results can be used to independently evaluate census coverage and content error, including reliability of selected characteristics collected in a census Incorporates matching of individuals or units between the census and PES Its results are generally more reliable than those of the census i.e. it justification for evaluation

27. Strengths and weaknesses of evaluation methods (contd.) Post enumeration survey Challenges: Requires highly skilled field and professional staff Matching is complex As it is supposed to be carried out immediately after the census at times there is lack of adequate funds to implement the PES exercise

28. Thank You!