1 / 27

“Advanced” Data Collection January 27, 2005

“Advanced” Data Collection January 27, 2005. Who We Are: Innovation Network. National nonprofit organization Committed to evaluation as a tool for empowerment Build evaluation capacity of nonprofits and funders so they can better serve their communities Practice a participatory approach.

ling
Download Presentation

“Advanced” Data Collection January 27, 2005

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. “Advanced” Data Collection January 27, 2005

  2. Who We Are: Innovation Network • National nonprofit organization • Committed to evaluation as a tool for empowerment • Build evaluation capacity of nonprofits and funders so they can better serve their communities • Practice a participatory approach

  3. Who We Are: Innovation Network www.innonet.org/services/LSC_Train

  4. Objectives of This Session • Review the five major groups of data collection methods • Discuss some of the subtleties of web-based data collection

  5. Data Collection Program Plan Evaluation Plan Data Collection Strategy

  6. Data Collection: 3 Steps • Choose the method • Decide which people or records will be the source of the information • Determine the level of effort involved in using that method with that population

  7. Data Collection Steps 1 & 2:Choose Method, Identify Source • Review documents • Observe • Talk to people • Collect written responses • Pictorial/multimedia

  8. Data Collection Step 3:Level of Effort • Instrument development • Cost/practicality of actually collecting data • Cost of analyzing and presenting data Also consider: • Communication Power • Proxy Power

  9. Document Review Examples: • Attendance records • Application information • Feedback forms • Web statistics • Curricula or content

  10. Document Review • Efficient and cost-effective • Dependent on the existence of documentation • Better for process evaluation than outcome evaluation

  11. Observation • When conducting and evaluation the observer: • Does not intervene in any way • Does not stand out or draw attention • Does not record names or any identifying information • It is important that observations be held over time (and at different places/days/times)

  12. Observation • Inexpensive way to obtain an outside perspective • Highly susceptible to bias • Subjective; relies on anecdotal data

  13. Interviews • Structured • Follow a predetermined interview protocol • Identify broad evaluation questions and specific interview questions in advance • Unstructured • Begin with one or two open-ended questions • Proceed based on the responses received

  14. Interviews • Provides the story behind someone’s experiences • Useful as a follow-up to a survey to further investigate responses • Time consuming (both in terms of collection and analysis) • Respondents susceptible to “leading” by interviewers

  15. Interview Tips • Make the respondent feel comfortable • Introduce yourself and the evaluation • Start with easy, short answer questions – end with more revealing, long answer questions • Record the interviews • Important for capturing great quotes

  16. Focus Groups A small group of people (who have something in common) who are led through a conversation by a moderator and discuss their opinions and provide suggestions on a topic.

  17. Focus Groups • Efficient and cost-effective way to gather qualitative data • Requires a skilled moderator/facilitator • Collects consensus rather than individual perspectives

  18. Focus Group Tips • Ideally 8-10 person groups • Maximizes amount of information you can gather without the suppression of voices • Don’t get bogged down • Rewards • Breakfast/Lunch/Dinner • Gift certificates

  19. Surveys • Delivery options: • By mail • In person • Online • By telephone

  20. Surveys • Collect quantitative information • Efficient • Open-ended questions (qualitative data) can be cumbersome for respondents and data entry

  21. Survey Tips • Introduce the survey • Describe the purpose and what will be done with the results • Reinforce that it is confidential and that you request they respond candidly • Lead with easy, closed-ended questions • Group categories of questions together • Consider how long it will take for someone to respond rather than the length of pages

  22. Web Statistics • # of hits • Most popular pages • Most popular downloads • Common “drop off” points • Where are they coming from?

  23. Don’t Be Afraid to Ask More! • Why are they there? • Who are they? What do they do? • What are they looking for? • Did they find it? • What are they going to do with it? • Are they willing to help you with future evaluation projects?

  24. Usability Testing We can help… ereed@innonet.org 202.728.0727 x 109

  25. Next Steps • We can do more! • Online, instructor-led trainings • Online, self-paced modules • Individual technical assistance www.innonet.org/services/LSC_Train

  26. Next Steps • We can do more! • Survey Builder • Survey Library • Data Analysis Tool

  27. Thanks for Your Participation! Measure results. Make informed decisions. Create lasting change. Innovation Network, Inc. 1625 K St. NW, 11th Floor Washington, DC 20006 (202) 728-0727 www.innonet.org Ehren Reed: ereed@innonet.org www.innonet.org/services/LSC_Train

More Related