Designing Surveys and Reporting the Results

1. Designing Surveys and Reporting the Results Workshop for Human Rights Researchers and Campaigners AAAS Scientific Responsibility, Human Rights and Law Program Eric Vance Director of LISA Assistant Research Professor Department of Statistics Material from this workshop will be available for download at www.lisa.stat.vt.edu/?q=workshops June 8, 2012 Washington, DC

2. Module 1: Introductions Key concepts you learned: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Questions you still have: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ~ Individual Exercise ~ Write down examples of using statistics/data in your work (especially surveys): ____________________________________________________________________________________________________ _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ~ Team Exercise ~ Share your best example with your team.

3. Module 2: Survey Fundamentals Key concepts you learned: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Questions you still have: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ~ Team Exercise ~ As a team, pick the answer choice you think would be the best method of data collection for the following scenario: The Millennium Challenge Corporation wants to know the impact on water quality and health of their program to drill borewells and install handpumps in villages in Mozambique that don’t have access to clean water. Convene focus groups of village women to ask them in-depth questions about the quality of their water and the health of their families before and after the installation of the handpumps (Before & After). Survey households in multiple villages about water quality and health (Before & After). Sample the old wells and the new handpumps and measure levels of pathogens, organic matter, dissolved O2, and salts in the water at the source and in the home. Examine local hospital records for incidences of diarrhea (Before & After). Use the results from a recently published study in a neighboring country to estimate the impact of the new handpumps.

4. Module 3: Questionnaire Design Key concepts you learned: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Questions you still have: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ • ~ Quiz ~ • Answer the questions below, first individually, then as team. Team answers should be recorded on the provided scratch-off answer sheet. • “Governments have a moral obligation to do what's best for the majority of their people and protect minority rights” is a double-barreled statement: • A. True B. False • Asking respondents their opinion on the statement above is likely to lead to • A. Selection biasB. Coverage error C. Nonresponse bias D. Measurement error • Which of the following is NOT a benefit of pilot testing a survey: • A. Check for selection bias B. Estimate variability in responses C. Reduce measurement error D.Change the wording of the questions E.None of the above • Open-ended questions are used more often than closed-ended questions in surveys because the responses are more genuine: A. True B. False • Which of the following factors into a sample size calculation: • A. Budget B. Time constraints C. Required precision • D. Expected variability of responses E. All of the above

5. Module 4: Survey Implementation Key concepts you learned: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Questions you still have: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ~ Team Exercise ~ Pick one example of an upcoming (or past) survey your team discussed after Module 1 and decide, as a team, how to implement this survey. If this was a past project, how would you modify it using what you’ve learned today? ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ __________________________________________________

6. Module 5: Reporting Results Key concepts you learned: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Questions you still have: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ~ Team Exercise ~ Which presentation of data or the reporting of a result in Module 5 was new to you and might be something you will use in the future? Discuss as a team.

7. Module 6: Questions and Answers ~ Individual Exercise ~ Review your questions from the previous modules and pick the one you most want answered. ____________________________________________________________________________________________________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ~ Team Exercise ~ Dr. Vance will answer one question from each team. As a team, decide what is your most pressing question, write it on a blank sheet of paper, and hand it to him. ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ __________________________________________________ __________________________________________________ __________________________________________________ __________________________________________________ __________________________________________________

8. EXTRA NOTES ____________________________________________________________________________________________________ ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ______________________________________________________________________________________________________________________________________________________ ______________________________________________________________________________________________________________________________________________________ ______________________________________________________________________________________________________________________________________________________ ______________________________________________________________________________________________________________________________________________________ ______________________________________________________________________________________________________________________________________________________ ______________________________________________________________________________________________________________________________________________________ ______________________________________________________________________________________________________________________________________________________