Census Evaluation Activities and Methods: Results and Lessons Learned

1. Results Generated from the Questionnaire Disseminated Prior to the Workshop

2. To better understand census evaluation activities at the country level To facilitate the sharing of experience on methods of census evaluation among countries in the region The Objective of the Questionnaire

3. Was PES conducted for 2000 census Yes : 6 No: 8  4 countries did PES for pre-2000 censuses  2 countries did not return the questionnaire All countries involve National statistical/population census offices in planning and conducting PES 6 countries where PES team different from census team (Australia, Bolivia, Brazil, Ecuador, Honduras, Peru) 3 countries used same team for planning census & PES (Panama, Uruguay, Venezuela)

4. Funding Planned as part of the census programmed in all countries except Venezuela Funding adequate in 6 countries (Australia, Bolivia, Brazil, Honduras, Panama, Venezuela)

5. Objectives of PES Census coverage Characterize missed or over-counted individuals Estimate undercount for use in estimating official resident population Learn quality of information collected – content error Establish ways to correct erroneous census information

6. Institutions responsible for PES and staff used National statistical office involved in all countries PES team different from census team in all countries except 4 (Panama, Paraguay, Uruguay, Venezuela)

7. Recruitment and training of PES staff Three countries selected enumerators and supervisors directly from those used in census (Ecuador, Honduras, Uruguay) Permanent staff used as supervisors in Australia and Uruguay Selection criteria includes: Location, experience, academic qualification Recruitment examination Training periods: 1-7 days

8. PES sampling designs Multi-stage, stratification, approaches Design stages: 1-3 (mostly 2) Australia: Three stages Bolivia: Two stages with sampling proportional to size Brazil, Peru: Two stages Ecuador, Panama: Two-stages with stratification Honduras, Paraguay, Uruguay: Stratification (one stage only)

9. Sample size for 1st stage units In many cases EAs were the primary sampling units Range of sample sizes  60 EAs in Honduras and 5000 EAs in Australia  1% of EAs in Uruguay and 20% of EAs in Panama  27642 individuals from 7340 households in Paraguay

10. Domains of analysis Wide range of domains used National; Rural; Urban Other domains  States/territories, region/province  Age, sex, marital status, urban/rural, cities  indigenous status, country of birth

11. Matching methods and documentation of rules Matching method Manual: 5 (Australia, Brazil, Honduras, Panama, Paraguay) Computer: 1 (Bolivia) Combination: 3 (Ecuador, Peru, Uruguay) Documentation of rules Yes:6 (Australia, Brazil, Ecuador, Honduras, Paraguay, Peru) No: 3 (Bolivia, Panama, Uruguay)

12. Field reconciliation visits Yes: 4 (Australia, Brazil, Ecuador, Honduras) No: 4 (Panama, Paraguay, Peru, Uruguay) Unknown: Venezuela

13. Coverage rate only (2 countries – Ecuador, Honduras) Net coverage rate (2 country – Paraguay, Peru) Coverage & net coverage rate (2 countries – Australia, Brazil) Omission rate (1 country - Bolivia) Panama did not calculate due to problems in methodology Some of the coverage measures used were

14. For content analysis the following were considered Estimating undercount and over-count by a variety of demographic categories Counts of misclassification between categories Rate of net difference Index of gross change Index of net change Percentage of identically classified Error of content Agreement

15. Producing report to inform lay users Planned to produce reports for lay users (5 countries - Australia, Bolivia, Brazil, Paraguay, Peru) Not planned to produce reports (Ecuador, Honduras, Panama, Uruguay) All countries that planned to produce reports did so Most countries 4-10 months from date of data collection to publication of PES report The longest period to produce report was taken by Ecuador (about 2 years)

16. Different interviewers and supervision arrangements (Australia, Brazil) No lists created in Census used in PES selection or interviewing processes Independent planning and execution (Bolivia, Ecuador, Honduras, Peru) Confidentiality on PES sample area (Brazil) Different periods (Uruguay) To achieve independence for PES:

17. Highlight of lessons learned Australia: evaluated the potential for computer matching To be introduced in the next round (subject to successful development) Bolivia: Methodology verification Brazil: Improve cartography and supervision; undercount rates are used in the next PES sample design Ecuador: Improve capacity and using imputation techniques

18. Highlight of lessons learned (continued) Honduras: Need for adequate capacity, independence, budget and logistical support Paraguay: Need for a separate budget, representative sample for each district and some outsourcing Peru: Need for larger PES coverage and budget Uruguay: Independence, high quality personnel, enough time and budget

19. 6 countries are planning to do PES (Australia, Brazil, Ecuador, Honduras, Paraguay, Uruguay) 5 countries are not planning to do PES (Bolivia, Chile, Costa Rica, Panama, Peru) Will measure coverage using demographic methods (Costa Rica) PES for 2010 round

20. Provide an independent check on Census coverage Identify characteristics of persons missed or over-counted Produce estimates of undercount for use in estimating the resident population Estimate content error (quality of information) Measure coverage and quality of information at province level and other domains Use results to correct census results, calculate population projections Some of the objectives mentioned for PES for 2010 round

21. Thank You END