Overview of Data Quality Issues in MICS

1. Multiple Indicator Cluster SurveysData Interpretation, Further Analysis and Dissemination Workshop Overview of Data Quality Issues in MICS

2. Data quality in MICS • Important to maintain data of the highest possible quality! • Important to examine data quality carefully before/during the interpretation of survey findings

3. Looking at data quality – Why? • Confidence in survey results • Identify limitations in results • Inform dissemination and policy formulation, avoid misleading policy makers, third parties • All surveys are subject to errors

4. Errors in surveys • Two types of errors in surveys • Sampling errors • Non-sampling errors

5. Sampling error • The difference between estimate and true value caused because the survey questions a sample of respondents rather than the whole population.

6. Non-sampling errors • Other types of errors, due to any stage of the survey process other than the sample design, including • Management decisions • Data processing • Fieldwork performance, etc • All survey stages are interconnected and play roles in non-sampling errors

7. Control of error in surveys Sampling errors can be estimated before data collection, and measured after data collection Non-sampling errors are more difficult to control and/or identify

8. Minimizing non-sampling errors in MICS • MICS has a series of recommendations for quality assurance, including: • Roles and responsibilities of fieldwork teams • Easy-to-use data processing programs • Training length and content • Editing and supervision guidelines • Survey tools • Failure to comply with principles behind these recommendations leads to problems in data quality

9. MICS data quality survey tools • Survey tools to monitor and improve quality, assess quality, identify non-sampling errors: • Field check tables to quantitatively identify non-sampling errors during data collection and to improve quality • Possible with simultaneous data entry, when data collection is not too rapid • Data quality tables to be produced at the time of final report

10. Data quality tables • A total of 28 tables • Data quality tables to look at: • Departures from expected (demographic, biological etc) patterns • Departures from recommended procedures • Internal consistency • Completeness • Indicators of performance

11. DQ.1 Age Distribution of Household Population Deficit at ages 0-1? Overall quality - heaping Heaping at age 5? Deficit – males AND females? More heaping at age 50 for females than males

12. DQ2. Age Distribution of Eligible and Interviewed Women Low response rates for women at young ages Surplus at age 50-54?

13. Might also want to look at the number eligible/number in the household list, by age DQ3. Age Distribution of Eligible and Interviewed Men Low response rates for men at young ages Surplus at age 50-54?

14. DQ.4 Age Distribution of Children Out-transference? Low response rates for infants? Out-transference?

15. DQ.5 Birth Date Reporting, Household Population Is the inclusion of question on date of birth justified?

16. DQ.5 Birth Date Reporting, Household Population Is the inclusion of question on date of birth justified?

17. DQ.6 to DQ.9 • More important to have full birth dates for individual respondents, adolescents, young people Birth Date and Age Reporting for women, men, under-5, and children, adolescents and young people – same structure

18. DQ.6 to DQ.9 • Target for these columns should be 100 per cent – especially for date of last birth, as it concerns eligibility, and is a very recent occurrence

19. DQ.11 Completeness of Reporting In general, target is to keep incomplete (missing, DK, etc) below 5 per cent Not for all types of information – especially those that relate to eligibility

20. DQ.11 to DQ.13 • Quality of anthropometric measurements • Proportion measured • Outliers • Incomplete date of birth

21. DQ.12 Quality of underweight data Should we actually use this data? Children excluded due to non-response or even incomplete date of birth may not be biased, but outliers is a big problem

22. DQ.13 Quality of stunting data Should we actually use this data?

23. DQ.14 Quality of wasting data Good data?

24. DQ.15 Heaping in anthropometric measurements Some heaping for height/length

25. DQ.15 Heaping in anthropometric measurements Usually, more heaping observed in length/height measurements than weight

26. DQ.16 to DQ.18 • Observations of birth certificates, vaccination cards and women’s health cards • Two “indicators” of data quality: • Performance of interviewers • Quality of information the survey collected

27. In all three tables, look for the proportion of existing documents the interviewers were able to see – as a performance indicator DQ.18 Women’s health cards Also look for the proportion of documents observed out of all under-5s or women – if these documents contain better quality information, that would be an indicator of overall quality of the data

28. DQ.19 Observation of bednets and places for handwashing Added complication of “moving kettles”

29. DQ.20 Person interviewed for the under-5 questionnaire Universally good data

30. DQ.21 Random selection of children Very significant improvement in the proportion of children correctly selected

31. DQ.22 School attendance by single age Cases should fall on the diagonal – look for outliers!

32. DQ.23 Sex ratio at birth Sex ratios among living children should be lower than for children deceased Should be around 1.02 to 1.06

33. DQ.24 to DQ.26 Tables on the quality of information collected in birth histories

34. DQ.24 Births by calendar years Important data quality indicator for birth histories

35. Check heaping – multiples of 7, days 0 and 1Percent early neonatal should increase by periodCompare with global numbers, earlier surveys

36. Check heaping – especially at 12 monthsPercent neonatal should increase by periodCompare with global numbers, earlier surveys

37. DQ.27 Completeness of information on siblings Missing information Missing information Missing information

38. DQ.27 and DQ.28 Mean sibship size should be increasing with age, due to falling fertility Look for sex ratios within normal ranges

39. Thank You