Multiple Indicator Cluster Surveys Survey Design Workshop

1. Multiple Indicator Cluster SurveysSurvey Design Workshop Sampling: Overview MICS Survey Design Workshop

2. Introduction • MICS : • Household survey program implemented across various countries and at multiple point in time within a country • Data are consistent and comparable across countries and over time

3. Contents • Importance of a correct sample design • Major steps in designing MICS sample • Key principles of MICS sample • Sampling options • Sampling tools available for countries • Group activity: sample size calculation

4. Sample Design • Sample design involves determining: • Sample size: number of units of analysis to be selected for the survey • Sampling structure: how those units are to be selected • Estimation procedures: how the results from the sample are to be used to draw inferences about the entire population of interest from which the sample was selected

5. Importance of a correct sample design • Sample size and structure affect: • Validity of inferences about the entire population • Magnitude of: • Sampling error • Non sampling error

6. Importance of a correct sample design • Sampling error: due to sampling of a small number of units from the population instead of complete enumeration • Non sampling error: due to problems during data collection and data processing (e.g., failure to locate and interview the correct sample household, misunderstanding of the questions, data entry errors)

7. Importance of a correct sample design • Link with other aspects of the survey • Dispersion of sample affects travel cost and time • Sample size affects the number of teams, interviewer workload, and cost of household listing and interviewing • Sample size affects timeliness of results • Large sample size can affect data quality

8. Major steps in designing MICS sample • Define objectives • Identify a suitable pre-existing sampling frame • Determine sample size and allocation

9. Major steps in designing MICS sample • Define objectives: • Key indicators • Reporting domains • Desired level of precision for survey results • Critical for sample size determination

10. Major steps in designing MICS sample • Identify a suitable pre-existing sampling frame: • Most recent census of population and housing • Master sample or sample for another survey conducted recently which is large enough to support the MICS sample design

11. Major steps in designing MICS sample • Identify a suitable pre-existing sampling frame: • The availability of a suitable sampling frame is a major determinant of the feasibility of conducting a MICS survey. • This issue should be addressed in the earliest stages of planning for a survey.

12. Major steps in designing MICS sample • Identify a suitable pre-existing sampling frame: • Regardless of source, evaluate the quality of the frame before drawing the sample • Characteristics of a good sampling frame • Complete coverage of the target population • No duplicates • Up to date

13. Major steps in designing MICS sample • Characteristics of a good sampling frame • Area units: Boundaries well defined and good maps are available • Identification codes • Measure of size (household or population) • Auxiliary information available for stratification

14. Major steps in designing MICS sample • Determine sample size and allocation • Survey objectives (key indicators, desired level of precision and need for sub-national results) • Sampling parameters from previous MICS or DHS (e.g., response rates, design effect) • Survey budget and resource constraints • Distribution of target population

15. Key principles in MICS sampling • Probability sample at every stage of selection (units are selected randomly with known and nonzero probabilities) • Latest census as sampling frame when available • Adequate sample size

16. Key principles in MICS sampling • Simple design • Sampling in two or three stages • Separate household listing • Clusters of moderate size: 20-25 households • No replacement of primary sampling units or households

17. Key principles in MICS sampling • Implement the sample exactly as designed • Proper sampling weights • Extrapolate survey results to the population • Used in all analyses to prevent biased results • Calculation depends on the exact sample design • Weights: households, women, men and children

18. Key principles in MICS sampling • Sampling error calculation • Possible only when probability sampling is used • Good sample documentation

19. Key principles in MICS sampling • Report on sample design describes: • Sampling frame • Sampling methodology • Sample size calculation and sample allocation • Survey domains and stratification • Probabilities of selection at each stage

20. MICS Sampling Option 1 – new sample with household listing • Design new MICS sample • Two stages with census as frame • Selection of census EAs with PPS at first stage • Carry out household listing in selected EAs/segments

21. MICS Sampling Option 1 – new sample with household listing • Select households systematically from listing • Interview selected households, no replacement will be allowed

22. Sampling Option 1 - continued • Advantages of option 1 - simple design - probability-based

23. Sampling Option 1 - continued • Limitations of option 1 - expense of listing households - time necessary to list households [Example, sample size of 5000 households may require 25000 to 50000 households to be listed]

24. MICS Sampling Option 2 – use an existing sample • Design MICS as a rider to another survey if timely and feasible • Use sample from a previous survey and re-interview households for MICS • Use old survey sample EAs and construct new listing of households to select for MICS

25. MICS Sampling Option 2 – use an existing sample • Old sample must be probability-based, national in scope • Possibilities – DHS, other national health survey, recent labour force survey • Important: design parameters must be known (such as selection probability, stratification, etc.)

26. Sampling option 2 - continued • Use of existing master sampling frame • Some countries use master sample design for intercensal national household surveys • Master samples generally sufficiently large for MICS; subsample of PSUs can be selected • Advantage – updated maps may be available for master sample of PSUs, and perhaps updated listing

27. Sampling option 2 - continued • Advantages of using previous sample - cost savings - maps available for interviewers - appropriate sampling plan available - simplicity

28. Sampling option 2 - continued • Limitations of using old sample - burden on respondents - sample design may need modification * sample size * sub-national coverage * number of PSUs or clusters • Balance between loss and gain

29. Sampling strategy for low fertility countries • In MICS 4 and 5, some low fertility countries are using second-stage stratification of listing by households with and without children under 5 • Higher sampling rate used for households with children • Increases number of households with children in MICS sample, and therefore number of sample children

30. Sampling strategy for low fertility countries (continued) • Improves the reliability of the child indicators without increasing the sample size to a very high level • This procedure also increases the variability in the weights and the design effects for the overall sample • Important to avoid very large variability in the weights for households with and without children • Differential weights between households with and without children generally should not exceed a factor of about 4

31. MICS Sampling Tools • Household listing manual and listing forms • Template for sample size calculation • Template for calculation of weights • Template for household selection • SPSS program for sampling error estimation

32. Sample Size Determination

33. Selection of key indicators • Choose an important indicator that will yield the largest sample size • Step 1: Select 2 or 3 target populations representing each a small percentage of the total population (pb); typically • Children 12-23 months: 2-4% or • Children under 5 years: 7%-20%

34. Selection of key indicators • Step 2: Review important indicators for these target groups but ignore indicators with very low or very high prevalence (less 10% or over 40%, respectively) • Do not choose from the desirably low coverage indicators an indicator that is already acceptably low • Do no choose childhood and maternal mortality ratios

35. n is the required sample size (number of households) • 4 is a factor to achieve the 95 percent level of confidence • r is the predicted or estimated value of the indicator in target population • deffis the design effect

36. RR is the response rate • pb is the proportion of the target subpopulation in total population (upon which the indicator, r, is based) • AveSize is the average household size (that is, average number of persons per household)

37. e is the margin of error to be tolerated at the 95% level of confidence • Currently, note that e = 0.12r [defined as 12% of r, in this case the relative standard error of r is 6% because e = 2 standard error (r)]

38. Previously in MICS2 • 2 different values for margin of error  • Margin of error was 5 percentage points for high values of r (over 25%) • Margin of error was 3 percentage points for low values of r (25% or less) • Difficulty for users in deciding on the sample size for their surveys.

39. MICS template for sample size calculation - EXCEL FILE