Tables on Sample and Survey Characteristics, Data Quality and Sampling Error

1. Multiple Indicator Cluster SurveysData Interpretation, Further Analysis and Dissemination Workshop Tables on Sample and Survey Characteristics, Data Quality and Sampling Error

2. Sample and Survey Characteristics • Response rates and background characteristics: • Set of 8 tables that: • Presents sample coverage and characteristics of households and respondents • Age and sex distribution of survey population • Characteristics of respondents • Household characteristics and wealth quintiles

3. Overall response rates are calculated for women, men and under-5's by multiplying the household response rate by the women's, men's and under-5's response rates, respectively.

4. Missing information on sex is normally not expected; in the event that few household members have missing sex in the final data set, this should be indicated in the final report in a footnote to the table, and such cases should be excluded from the table.

5. Total weighted and unweighted numbers of households should be equal when normalized sample weights are used. Tables HH.3, HH.4, HH.4M and HH.5 present main background characteristics of the household, women's, men's and under-5 samples, and should be produced and finalized before the rest of tables are produced, to ensure that the categories adopted for presentation in the tables will include sufficiently sized denominators. Religion/Language/Ethnicity of household head should be constructed from information collected in the Household Questionnaire, in questions HC1A, HC1B, and HC1C. In most surveys, some combination of these three questions will be used as the final variable that best describes the main socio-cultural or ethnic groups in the country.

6. Information on housing characteristics are obtained in the Household Characteristics module of the Household Questionnaire: Electricity (HC8A), flooring (HC3), roof (HC4), exterior walls (HC5) and rooms used for sleeping (HC2). To limit the size of the table, detailed floor, roof, and exterior wall categories are not shown. If needed, these categories may be indicated in a footnote below the table, in the final report. Additional relevant housing characteristics may be added to the table if included in the household questionnaire. Most of the information collected on these housing characteristics are used in the construction of the wealth index.

7. Information on household and personal assets are obtained in the Household Characteristics module of the Household Questionnaire: Radio (HC8B), television (HC8C), Non-mobile telephone (HC8D), refrigerator (HC8E), agricultural land (HC11), farm animals/livestock (HC13), watch (HC9A), mobile telephone (HC9B).bicycle (HC9C), motorcycle or scooter (HC9D), animal-drawn cart (HC9E), car or truck (HC9F), and boat with a motor (HC9G). Ownership of dwelling is based on responses to HC10. Additional household and personal assets should to be added to the questionnaires (for wealth index construction) and shown in this table. Missing/DK values are included in the denominators and households with missing information are considered not to own or have these assets. However, a careful examination of the extent of missing values needs to be undertaken prior to the construction of this table. If Missing/DK cases exceed 5 percent, this should be shown in the table. Most of the information collected on household and personal assets are used in the construction of the wealth index.

8. Wealth index quintiles are constructed by using data on housing characteristics, household and personal assets, and on water and sanitation via principal components analysis. Household members should be equally distributed to the five wealth index quintiles for the total sample, in the first row of the table (percentages that deviate from the equal distribution of 20 percent per quintile by 0.1 - 0.2 percent are permissible). Other background characteristics (such as Religion/Language/Ethnicity, education and sex of household head) may be added to the table, if needed.

9. Data Quality Tables • Before producing tabulations and writing the report narrative, 28 tables are produced for assessment of data quality • Intended to check distributions, heaping, understatement or overstatement, sex ratios, eligibility and coverage, out-transference of eligible persons, the extent of missing information, outliers, sex ratios, quality of anthropometric measurements • Useful for understanding quality issues, familiarity with issues in data sets, indicative of the quality of training and implementation

10. If age reporting is good, the distribution should be smooth. The table should also provide insights into overreporting or underreporting at certain age groups or intervals, and the extent of missing information on age. Deficits at ages 4, 15, and 49, excesses at ages 5 and 6, 14, and 50 might be indicative of out-transference of ages to avoid administering individual questionnaires.

11. Distribution of household members by single age

12. Table DQ.2: Age distribution of eligible and interviewed women The purpose of these tables is to detect both displacement of respondents out of the eligible age range and differential response rates by age.

13. Completion rates - women, men & under-5s (DQ2, DQ3, DQ4) Fieldwork performance – re-visits, good planning Completion rates need to be high, but also uniform by age and background characteristics Low completion rates for certain age groups are likely to bias results

14. Birth date and age reporting (DQ5, DQ6, DQ7, DQ8, DQ9, DQ10) Surveys always have cases with missing information The extent of missing information is important, because it can result in biased results if such proportions are high Particularly informative about the quality of survey is the extent of missing information on measurements, ages, and dates of events

15. DQ.5: Birth date reporting: Household population

16. Completeness of reporting (DQ11) The purpose is to examine the amount of missing information for certain key indicators. High levels of missing data may indicate that the non-missing data are biased or of poor quality.

17. Completeness of anthropometric data (DQ12, DQ13, DQ14) • Many tools have been developed for assessing data quality of anthropometric indicators • Completeness of anthropometric data influenced by • Birth date reporting • Children not weighed, measured • Bad quality measurements • Expected completeness should be above 90 percent, preferably 95

18. Completeness of anthropometric data (DQ12) - Underweight

19. Heaping in anthropometric data (DQ15) Under normal circumstances, approximately 10 percent of anthropometric measurements should be reported for each of the digits for the decimals. Significant excesses over 10 percent are indicative of heaping, and therefore quality problems in anthropometric measurements, either due to truncation or rounding. Typically, more heaping is expected in height/length than weight measurements.

20. Heaping in anthropometric data (DQ15) The table includes all children with weight and height/length measurements, regardless of the completeness of date of birth information, and flagged cases, which may not be included in the anthropometric analysis.

21. Observation of documents (DQ16-DQ18) and observation of bednets and places for handwashing (DQ19) Interviewers are required to ask and see the specific documents and copy relevant information on the questionnaire This is important for the completion of the several modules in women and under-5 questionnaire, and may also be useful for obtaining accurate information on children's dates of birth and ages

23. DQ20: Respondent to under-5 questionnaire • Presence of mother in the household and the person interviewed for the under-5 questionnaire: • The under-5 questionnaire should be administered to the mother, if the mother is listed the household roster

24. DQ21: Correct selection for child labourand child discipline modules • Selection of children age 1-17 years for the child labour and child discipline modules • In households where 2 or more children age 1-17 years live, interviewers are required to select, according to pre-determined random selection procedures, one child for the child discipline module • Percentages with correct selection should be close to 100.0

26. DQ.22: School attendance by single age Age at the beginning of the school year is calculated from dates of birth of household members or by rejuvenating household members based on the date of the survey and current age. Levels and grades refer to the current school year, or the most recent school year if data collection was completed between school years. Many cases outside the diagonal would be indicative of both poor fieldwork supervision, as well as poor data entry and (lack of) verification.

27. Child mortality related (DQ23-DQ26) DQ.23: Sex ratio at birth among children ever born and living DQ.24: Births by calendar years DQ.25: Reporting of age at death in days DQ.26: Reporting of age at death in months

28. DQ.23: Sex ratio at birth among children ever born and living

29. DQ.24: Births by calendar years The purpose is to examine the impact of omission of births in the five years preceding the survey. If large amounts of omission are suspected, then careful interpretation of current fertility and mortality levels and trends is needed. Graphic presentation of these data can provide good visual appreciation of omission and transference.

30. The purposes of tables DQ25 and DQ26 are to examine the possible omission of neonatal and early neonatal deaths; and the effects of age at death heaping.

31. Maternal mortality related (DQ27 and DQ28) DQ.27: Completeness of information on siblings DQ.28: Sibship size and sex ratio of siblings

32. Sampling error tables

33. Sampling Error Tables: Background • The sample selected in a survey is one of the many samples that could have been selected (with same design and size) • Sampling errors are measures of the variability between all possible samples, which can be estimated from survey results

34. Sampling Error Tables: Background • Calculation of sampling errors is very important • Provides information on the reliability of your results • Tells you the ranges within which your estimates most probably fall • Provides clues as to the sample sizes (and designs) to be selected in forthcoming surveys

35. Sampling Error Tables: Background • MICS sample designs are complex designs, usually based on stratified, multi-stage, cluster samples • It is not possible to use straightforward formula for the calculation of sampling errors. Sophisticated approaches have to be used

36. Sampling Error Tables: Background • Versions 13 and above of SPSS are used for this purpose • SPSS uses Taylor linearization method of variance estimation for survey estimates that are means or proportions • This approach is used by most other package programs: Wesvar, Sudaan, Systat, EpiInfo, SAS

37. Sampling Error Tables: Background In MICS, the objective is to calculate sampling errors for a selection of variables, for the national sample, as well as for each of the reported domains Sampling error tabulation plan includes separate excel worksheets for: total sample, urban, rural, and 6 regions. SE tables can be produced for other domains such as ethnicity and wealth quintiles

38. The indicators listed in SE tab plan represent the MDG indicators for which SEs can be calculated. SEs can easily be produced for most other MICS indicators and included if desired. Note that mortality SEs can only be calculated for results based on birth history with the existing and separate SPSS syntax. Also note that SEs for the maternal mortality ratio can be calculated only through the CS Pro application.

39. Comprehensive knowledge about HIVprevention among young people

40. Thank You