Objectives and data needs

1. Objectives and data needs (Session 01)

2. Module overview • Basic concepts and definitions • Sampling methods – simple random, stratified, cluster, multi-stage, etc • Designing a sampling scheme for relatively simple scenarios in accordance with objectives and available resources • How to produce estimates for population characteristics with measures of precision • Sample size determinations • An appreciation of what is meant by sampling weights

3. Module aims By the end of this module, you will be able to • explain what is meant by sample, population, sampling frame, sampling units • explain the notions of representativeness and generalisability of results • design sampling schemes for simple scenarios • produce population estimates and associated measures of precision • discuss options for calculation of sample sizes with a good understanding of general issues involved

4. Aims – this session… By the end of this session, you will be able to • appreciate the different type of objectives that may arise in real life surveys • critically assess the type of data needed to address questions of interest • explain the benefits of sampling • recognise importance of utilising existing knowledge about the population sampled We begin with some general remarks, then move to survey objectives & other issues.

5. Some general remarks • This module is on sampling ideas and issues arising from sampling procedures when conducting a survey • It is not about survey methods and analysis – which are covered in Module H7 • As such, there will not be coverage of the survey process except where it relates to sampling • The accompanying handout gives an outline of the survey process so the context is clear – useful to read…

6. Survey Objectives • Decisions regarding the sampling process cannot be made rationally unless we are clear about the survey objectives. • Surveys conducted by national statistical offices are often done to provide information on which policy decisions can be made • The stated objectives are often vague, e.g. “the objective of this survey is to collect information about…….” • OK as a starting point, but need to be more specific if a sensible sampling procedure is to be used – some examples follow…

7. Objectives related to estimation To estimate • median income of dwellings in slum areas of a city • proportion of rural households that have no access to a medical facility within 3 kms • maternal mortality rate, i.e. deaths per 1000 live births of mothers from puerperal causes • mean yield per hectare of pigeonpea production in small-holder commercial farms

8. Objectives related to comparisons Questions of interest may be: • does a newly introduced farming practice for managing banana plants result in higher yields compared to a standard practice? • is there a difference in access to health facilities between rural & urban areas? • is there evidence that children from poorer families have less opportunities for entering higher educational institutions?

9. Objectives related to relationships Is there a relationship between • consumption expenditure (as a proxy for income) and household demographics and assets? • children’s enrolment in primary school and educational level of household head? • mean number of visits by household members to a health clinic and their level of access to clean water and adequate sanitary facilities?

10. Sampling units and data Sampling is a first step in any survey study. There is always a need, before data collection, to identify (amongst other things):- • the ultimate sampling unit on which measurements are to be made • actual measurements needed, plus clarity on the calculation of any derived variable(s) • sampling procedure to use to select sampling units Two examples from slides 7 and 9 follow…

11. Initial steps – some examples Example 1. Estimating the proportion of rural households that have no access to a medical facility with qualified personnel within 3 kms of their homestead. Unit:Household Measurement:Distance to the closest medical facility (latter appropriately defined) Derived variable:Coded as 1 if above measurement > 3 kms, 0 otherwise

12. An example with a hierarchy Example 2. Determining if there is a relationship between number of visits by household members to a health clinic and their level of access to clean water and adequate sanitary facilities? Units (within HH): Household members Measurements:Visits made by each member in HH, how clean water is accessed, sanitary facilities Derived variable: Mean number of visits = Sum of visits by all members divided by HH size (needed as objective is at the HH level, but measurement is at person level)

13. Selection of households? • Another aspect needed before data collection is to plan the sampling procedure, i.e. how the households can be selected. • Here it is important to study the sampling setting to understand better the structure (social, geographical etc) of the “target” population from which households are to be drawn. • Here “target” population refers to the group of households to which the survey results are intended to generalise.

14. Using existing knowledge • Examining the literature should also point to existing data sources – need to avoid duplicate data collection. Use what is relevant. • Also use general knowledge about the target population. Often much is on record: it would be desirable to use this information • Administrative structure: e.g. districts, counties, subcounties, parishes, villages in Uganda • Agro-ecological regions • Rural/urban divide, etc • Build knowledge about existing data sources and population into the sampling and data collection process

15. Why Sample? • The whole population is rarely measurable • An exception is the census, e.g. usually population censuses are done once every 10 years • A well-designed sample enables us to extrapolate our results to the population • Statistical methods enable us to measure the reliability of our conclusions Though they were covered in Module H2, we reiterate benefits below for completeness and as a reminder.

16. Benefits of sampling • Cheaper, quicker and administratively easier than census • Less prone to errors – and those that do occur are more easily identified • A well-thought out sampling procedure can ensure proper coverage of major population characteristics • If suitably structured, the sample survey can (i) take account of varying sizes of units, e.g. farms, and (ii) correct for under-enumeration and some sorts of non-response.

17. Limitations of sampling • Sound sample surveys require considerable time and effort to plan and run. If tasks entailed and resources needed are under-estimated, the results will be poor • Unless a pre-determined data analysis plan is in place at the start, data relevant to objectives may not be collected, or too much unnecessary data will be collected • Training survey staff is crucial. Ill-phrased questions, poorly linked to objectives, can lead to non-informative results

18. References De Vaus, D.A. (2001) Research Design in Social Research. Sage Publications, London. ISBN 0 76195346 9 KALTON, G. (1990) Introduction to Survey Sampling. Sage Publications. ISBN 0 8039 2126 8