1 / 25

Impact Evaluation: Initiatives, Activities, Coalitions

2. Objectives. Participants will learn:1. When an evaluation should be conducted2. What questions to ask to address outcomes

colum
Download Presentation

Impact Evaluation: Initiatives, Activities, Coalitions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Impact Evaluation: Initiatives, Activities, & Coalitions Stephen Horan, PhD Community Health Solutions, Inc. September 12, 2004

    2. 2 Objectives Participants will learn: 1. When an evaluation should be conducted 2. What questions to ask to address outcomes & impact 3. Strategies for evaluating outcomes & impact 4. Strategies for incorporating multiple programs and activities 5. Strategies for incorporating community input

    3. 3 Ground Rules Evaluation talk can be confusing dont let semantics get in the way This presentation defines Outcomes as changes in health program clients Impacts as changes in the organization, system, or community Different funders or textbooks might define these terms differently This presentation uses a model of personal change based on knowledge, attitudes & beliefs, skills, behaviors, and health status. There are other equally valid models no one is saying you have to use this one!

    4. 4 When to evaluate 1. When a funder requires it 2. When the program is experimental 3. When the program is controversial 4. When the program is expensive 5. When you think the program might not be worth it 6. When you think the program is worth it but might be in danger of losing funding

    5. 5 Questions for addressing outcomes & impact

    6. 6 Questions for addressing outcomes & impact Five phases of evaluation 1. Context evaluation 2. Program design evaluation 3. Program implementation or process evaluation 4. Program outcome evaluation 5. Program impact evaluation

    7. 7 Questions for addressing outcomes & impact 1. Context evaluation Why is this program needed? Does this program duplicate other programs? Does this program conflict with other programs? How might public policy affect this program? Does this program fit our funders criteria?

    8. 8 Questions for addressing outcomes & impact 2. Program design evaluation What is our program theory? Who is our target population? What are the needs of our target population? What outcomes will we help out target population achieve? What services will we provide? How many people will we serve? What resources will we need to run the program?

    9. 9 Questions for addressing outcomes & impact 3. Program implementation evaluation How many people are we serving? What services are they receiving? Are they receiving the services they should? What activities are we conducting? How much are we spending? All of the above compared to Program Design

    10. 10 Questions for addressing outcomes & impact 4. Program outcome evaluation How have our clients changed in terms of: Knowledge Attitudes & beliefs Skills Behaviors Health status

    11. 11 Questions for addressing outcomes & impact 5. Program impact evaluation How has our organization or community changed in terms of: Financing Service capacity Organization of services Quality of services Accessibility of services Cost of services Community health status

    12. 12 Strategies for evaluating outcomes & impact

    13. 13 Strategies for evaluating outcomes & impact 1. Begin with your program theory 2. Develop your program design 3. Capture your program design in a logic model 4. Decide which outcomes & impacts to measure 5. Negotiate outcomes & impacts with your funder

    14. 14 Strategies for evaluating outcomes & impact 1. Begin with your program theory Program theory is your set of assumptions about cause and effect between our program and are target populations outcomes Most often overlooked component of program evaluation What makes you think this will work?

    15. 15 Strategies for evaluating outcomes & impact 2. Capture your program design in a program logic model Target population Assumptions (program theory) Inputs Activities Outputs Outcomes Impact

    16. 16 Strategies for evaluating outcomes & impact 3. Develop your program design Specify your target population Specify your program theory in a set of assumptions Specify your target impact Specify your target outcomes Specify your target outputs Specify your required activities Specify your required resources

    17. 17 Strategies for evaluating outcomes & impact 4. Decide which outcomes & impacts to measure Is this outcome or impact important to our audiences? Is this outcome or impact within our sphere of influence? Is this outcome or impact achievable within the timeframe of the reporting period? Is this outcome or impact measurable?

    18. 18 Strategies for evaluating outcomes & impact 5. Negotiate with your funder Not every outcome or impact can be measured Not every outcome or impact should be measured Arm yourself with knowledge Your program theory Why a particular outcome or impact should or should not be measured What it will cost to measure a particular outcome or impact If they insist, tell them what you think the evaluation will cost, and ask them to either fund it or suggest a cheaper way to do the evaluation

    19. 19 Strategies for incorporating multiple programs and activities

    20. 20 Strategies for incorporating multiple programs and activities 1. Get everyone to agree on: Program theory Target outcomes Target impacts 2. Get everyone to develop a program design complete with logic model 3. Agree upon common outcome indicators 4. Agree upon common impact indicators 5. Share data collection tools 6. Share data analysis methods 7. Pool the data or agree on a standard format for aggregating the data

    21. 21 Strategies for incorporating community input

    22. 22 Strategies for incorporating community input 1. Ask community members to review and comment on your program logic model 2. Ask community members to review and comment on your evaluation workplan 3. Ask community members to pilot test your data collection instruments 4. Survey your community on its perceptions of your program 5. Ask community members to help you interpret your evaluation results

    23. 23 Additional resources

    24. 24 Additional resources For additional ideas, guidelines, and connections, go to www.CommunityHealthResource.net In particular: Go to the Program Design page and Program Evaluation page and download worksheets

    25. 25 Presenter Stephen Horan, PhD President Community Health Solutions, Inc. 9603 BC Gayton Road Suite 201 Richmond, VA 23238 T: 804.673.0166 F: 804.673.2244 shoran@communityhealthinfo.com

More Related