Sampling Considerations, Including Using a PDA with GPS to choose a sample and conduct a survey.

1. Sampling Considerations, Including Using a PDA with GPS to choose a sample and conduct a survey. Statistical and Spatial Considerations Allen Hightower RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

2. Sampling Design – Considerations: 1 • Precision at the national and first level administrative units. • The ability to evaluate large scale programs • National in scope • First admin level • Not the second admin level! • The ability of the survey to give answers to your tabulation plan (mock report). RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

3. Sampling Design – Considerations: 2 • Appropriate sampling frame • Is your target population correct? • Survey the whole country or only areas with malaria risk? • Cost benefit /logistics • Should you include large/remote areas with little population? • Eg, NE province in Kenya: large, arid area • Political Considerations • Program evaluation • Ethnic groups of interest RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

4. Sampling Design – Considerations: 3 • Usually use a multi stage survey with a combination of the below components: • Simple Random Sample • Best precision, valid • Systematic sample….actually is a cluster sample. • Cluster Sample • Less precision, valid. Operationally easier to implement. • Multi-stage clustering: potential analytic issues • Stratified sample: to increase precision. Often used to define domains of interest. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

5. Sampling Design – Considerations: 4 • Be sure that your sampling plan will cover domains/target populations of interest: • Rural/urban : 75/25? • Malaria endemic zones • Pregnant women: This was the hard one for Kenya • Achieved by increasing cluster size • CU5s RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

6. Sampling Design – Considerations: 5 • What do you do if your sampling frame is out of date or does not exist? • Work with the CSO to develop a new one. • Work with CSO to improve the existing one. • Two tracks: • Paper: the household listing system • PDA: Use GPS Survey RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

7. Why Use GPS Survey? • CSO maps often out of date • Peri urban areas • Urban areas • All of them? • GPS survey creates an up to date sampling frame (ie, HH list) • GPS survey creates a statistically valid sample RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

8. Why Use GPS Survey? • GPS survey can be used in cluster or simple random samples at the last stage. • GPS Survey can provide information on the interviewer not possible with paper-based surveys. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

9. PDA data collection: How safe? Data saved field-by-field • Battery Loss – non-issue. • Memory is nonvolatile like a digitall camera. • Battery life >>16 hrs • Better than laptop/tablet computers • Big screens, HDs use power! • No moving parts (ie, better than a laptop) • Encryption/PW at OS level • PDAs we use are very, very sturdy. • Have never had one stolen/lost. • CDC Kenya collected > 12 million pieces of paper electronically with PDAs last year. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

10. PDA data collection: Can Paper do this? • Live skip patterns – less training • Only see the right questions. • QC at Data Collection point • Autoword completion: less typing • Turn off unneeded features (games/buttons) • Know where interviewer is/should be • Know when the interview started/stopped • Very, very rapid data availability RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

11. Concept of PDA-GPS system • Interview teams take nothing more than PDAs with attached GPS units into a cluster • Map HHs in the cluster • Choose a statistically valid sample • Conduct Software Guided Interview w/PDA • Data Rapidly Available for Reporting/Spatial Analyses RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

12. PDA-GPS mapping system • Simple, fast • Most clusters mapped in 4 hrs or less • Village Name (auto carry) • HH number (auto increment) • Comment (e.g., the house in the middle, family name) • GPS gets you close, Comment provides final ID info • Allows Multiple mappers RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

13. GPS Survey Program:GPS readings • Program checks the CF-GPS unit for a valid reading • Gives color coded GPS validity • Green – good precision • Yellow – marginal • Red – poor • Random number assigned to HH when data are saved. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

14. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

15. Sampling Design • It is your decision! • The system only affects how households or compounds are sampled within a cluster. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

16. GPS Survey-Sampling: • You tell the PDA program : • How many HHs to sample from the EA. • How many alternates HH’s? • SRS or Cluster Sample • Parameters set up in XML file • You get: • a random sample where each HH has a known probablity of selection. • A File of the GPS data for all mapped HH’s and features of interest • Denotes which HH’s are in the sample RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

17. GPS Navigation Software • Guides the interviewer back the sample HH • List of HH ID’s with distance and compass direction /pointer • Also a map of HHs • Can mark HH off list • Can divide a list RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

18. GPS Navigation Software • List of HH ID’s with distance and compass direction/pointer • Can mark HH off list • Can divide a list to allow multiple interviewers to split work. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

19. GPS Navigation Software • Also a map of HHs • Can show HH ID info RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

20. More to this than technology!!! • This is a system that uses technology. • Software development and testing. • Training, training. • Stay in touch with the data collectors in the field regularly. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

21. Training!!! • Can train people to use a lousy system • Poor training can ruin a perfect system • Learn how to solve common problems • Tip sheets/manuals • Training schedules/exercises • Stay in touch with the data collectors in the field regularly. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

22. Advantages of GPS-based Simple Random Sample • Gold Standard Approach • SRS – better precision than other approaches. • Unbiased (accurate) • Any HH in village can be a part of sample • You know the probability of selection for a HH. • IE, a probability sample. • Spatial Analyses RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

23. Why is “Statistical Validity” Important? • For any survey, even those not using statistically valid methods, you can compute means, proportions, std errors. • However, to know the statistical distribution and compute confidence intervals for the means and proportions, you MUST use a random sampling process. • Or you must pretend that you used a random sampling process. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

24. How much work to map cluster? • 4 hours to map the EA, typically • Some take 2 days! • 1 to 3 minutes between GPS readings (between HH’s) • Each interviewer mapped between 30 to 45 HH’s per EA RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

25. GPS can be used for quality control. Here we caught someone standing in one place. All GPS readings were within 60m. Sept 2005 Bad (Red): Green =remapping of village! RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

26. GPS can be used for quality control. In the Kenya MIS, we had an “extra” cluster mysteriously appear. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

27. As we looked, the phantom cluster consisted of points that belonged to other 4 clusters. All pts were very close together. Turns out, the team had stopped at a meeting place to “complete their quota.” Our system has an audit trail, taking a GPS reading each time a household record is changed. RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

28. Can Produce a Map of HH’s in each EA,Showing which were in the Survey • Survey Program produces Database file with: • Lat/Lons for all HH’s in EA • Indicator for whether HH was included in sample • Lat/Lons for all other features of interest • Easily Imported into other GIS products RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

29. Standardized data collection tools • MIS’s in Zambia, Kenya, Ethiopia • Bednet Ownership/Usage surveys in a number of countries. • Comparable methods, systems • Extremely cost effective if systems and pdas are used multiple times. • Not sure it is worth it to do just once RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia

30. Thank you! RBM-MERG Malaria Indicator Training Workshop, 9th-12th September 2008, Lusaka, Zambia