Statistical Office of the Republic of Serbia

1. Post enumeration survey in the 2009 Pilot Census of Population, Households and Dwellings inSerbia Olga Melovski Trpinac Statistical Office of the Republic of Serbia www.stat.gov.rs pub@stat.gov.rs

2. CENSUS POST ENUMERATION SURVEY The post enumeration survey (PES) has a long tradition in Serbia. It has been conducted after each census since 1953. The main goals of the survey have been: • to estimate coverage errors in enumerating population, households and dwellings; • to evaluate quality of answers to selected census questions; • to point out ways to improve future census design and operations.

3. PILOT CENSUS POST ENUMERATION SURVEY (PPES) • For the first time in 2009, census post enumeration was a part of pilot census. • It was conducted immediately after the 2009 Pilot Census (PC) in the period from 22 to 30 April on 11 enumeration areas (EAs) with about 850 dwellings and 2000 persons. • It is based on fundamental methodology principals already set up in census PES survey in Serbia, such as independence between the post enumeration survey and census. • The main difference to the actual PES concerns the sample of EAs. Contrary to the census PES sample, the sample of EAs is small and not random.

4. GOALS Estimates of coverage errorsand quality of answers could not be obtained but it is possible to: • collect experiences regarding the adequacy of basic tools and procedures used in the survey; • obtain knowledge of local conditions; • test particular census questions. The outputs of these activities will be used to improve the quality of 2011 Census and Post Enumeration Survey.

5. PHASESOFTHE2009 PPES Activities prior to the collection of data in the field are: • Budget planning • Methodology development • Sample selection • Questionnaire design • Preparation of the Manual and training of the post enumerators

6. PHASES OF THE 2009 PPES(cont.) Activities that follow collection of data are: • Manual matching, linkage of the PPES data to the PC data. • Comparison of PPES and PC data (copying data from PC questionnaires to post to PPES questionnaires). • Data entry program development. • Data entry. • Data analysis and summary of the results. • Integration of results into a List of recommendations for the 2011 Census.

7. RESULTS OF THE 2009 POST ENUMERATION SURVEY • All phases of the survey have yet to be completed. • There are already outcomes that are valuable inputs for preparation of 2011 Census and PES tools and operations.

8. TRAINING • The PPES served as a training vehicle for new staff that will be engaged in 2011 PES. They were given a chance to take part in all phases. • In that way their awareness of various sources of non sampling errors would be increased as well as the understanding of the work ahead in 2011 Census. • In addition, team work and discussions concerning PPES operations resulted in new ideas and suggestions of how to introduce changes in census PES in order to improve the quality of the survey.

9. THE BASIC QUESTIONNAIRES They were designed to be as similar as possible to the original PC questionnaires but with columns where the census data can be copied after PPES data collection. They are: • The Questionnaire for Enumerating Dwelling, Household and Persons (QE) and it is used for collecting coverage data on dwellings, households and persons in selected EAs. • The questionnaires: • For Household and Dwelling and • Individual Form. They are used to collect data that is needed for evaluation of quality of answers to selected census questions. Experience in this PPES indicate necessary changes of the QE in order that the data is more easily filled and entered.

10. CONTROLLER FORM Controller form, similar to the one in PC, was introduced for the first time in this PPES. The post enumerator was obliged: • while in the field, to fill a table row for each dwelling or household with: the address information, identification of dwelling and household, name of the head of the household and the number of persons in the household and temporarily present; • after collecting data in EA, to fill the front page fields with summary data for EA; • in the phase of comparison, to copy on the front page fields corresponding PC summary data.

11. CONTROLLER FORM (cont.) This form has been found very useful: • for systematic selection of households for quality check of answers; • for supervision of the post enumerators’ field work; • in linking the post enumeration data to census data. Also, the summary data on the front page of this form could replace the data of one of the auxiliary forms, Comparative list, that was used in previous census PES.

12. THE MANUAL AND THE TRAINING OF POST ENUMERATORS According to experience in the PPES: • the Manual and the training for 2011 PES should include only instructions that are specific for the survey; • the post enumerators should be chosen among the best census enumerators. For the success of the PES, it is crucial to stress the significance of independence of census and post enumeration. The post enumerator must collect data in the EA he/she is in charge of without seeing the data collected in census for that particular EA.

13. COLLECTION OF DATA Already known causes of errors were encountered. In order to reduce them in 2011 PES it is important that: • up to date maps of EAs with clear descriptions are prepared; • post enumerators have perfect acknowledgment and understanding of census definitions; • post enumerators establish an adequate contact with the respondent; • population is notified through various media and on time of the PES, in selected EAs.

14. MATCHING AND COMPARISON OF PPES DATA TO THE PC DATA From experience in the PPES, it is necessary that the Manual and training are improved for these complex activities: • More attention should be dedicated to ambiguous cases such as: unit not registered in census or post enumeration, wrongly enumerated, household splits and merges. • It is recommended that the 2011 census PES training includes a test with sample data for matching post enumeration to census data and copying census data to PES forms.

15. DATA ENTRY APPLICATION AND ENTRY • Data entry application has been developed in CSPro. This application is complex and it enables multilevel data entry of the basic PPES questionnaires. • For coded questions range errors can not be made during data entry, but missing and skip errors and any inconsistencies are allowed as long as they are on the questionnaire being entered. • The data entry quality is much more important than speed.

16. ANALYSIS • The analysis of data begun with comparison of 17 answers to questions of the individual forms that were filled for 180 persons in PPES and PC. (From 10 per cent subsample of households selected for quality check from 10 EAs chosen for PPES). • If there was a difference to an answer to a particular question, the material collected in Pilot Census was examined to check whether the copying has been done correctly.

17. ANALYSIS (cont.) • Out of all possible answers (180 x 17 = 3060), 7.7 per cent are different and 0.7 per cent of differences are due to incorrect copying. • Out of 180 questionnaires 39 per cent are without error (complete agreement of post enumerated and census data), 31 per cent with 1 discrepancy, 16 per cent with 2, 6 per cent with 3 differences and 7 per cent with more than 3 discrepancies.

18. ANALYSIS (cont.) Questions that are most frequently answered differently are those: • that demand two answers; • need to be filled according to some additional instruction on the questionnaire; • hard to answer because they are personal. Causes for differences could also be: • carelessness in collecting data; • different respondent in PC and PPES.

19. ANALYSIS (cont.) • Questions that are most difficult for copying are those: • with a longer list of possible answers; • the question concerning the 6-digit identity number. • Household data has also been analyzed. • Analysis of coverage it not yet completed.

22. METHODOLOGICAL ISSUES In the phase of copying census data to PES forms, it is suggested to copy to the QE only necessary identification of: • dwelling; • household; • persons; • the name of person (if a person is not registered in PES but registered in the census). • All other census data can be obtained by merging PES with census data, after completing census and PES data entry.

23. METHODOLOGICAL ISSUES (cont.) • A precondition for this procedure is that census and PES data entry programs respect the same rules. • In this way: • copying of data would be avoided, time saved and errors caused by copying reduced; • PES data would be compared to census data after data entry. This is data used in further processing so the procedure would make sense. • Acceptance of this idea still has to be considered. Among other factors, it depends upon decisions concerning 2011 Census data entry (data entered by operators or scanned).

24. CONCLUDING REMARKS • All phases of the 2009 PPES are not yet completed. • There are already many useful findings that will help to improve the methodology, organization, collection and processing of data in 2011 Census PES. • It seems that conducting a pilot post enumeration survey presents an excellent way to prepare for the census PES.

25. REFERENCES: • Zarkovich, S.S. (1966) Quality of Statistical Data, Food and Agriculture Organization of the United Nations, Rome. • 2009 Pilot Census of Population, Households and Dwellings - Instructions for enumerators, questionnaires and database. • Pilot Census Post Enumeration Survey - Instructions for post enumerators, questionnaires and database.

26. Thank you for you attention!