1 / 18

The Big Bang: Creating an Evaluation Program From the Beginning

The Big Bang: Creating an Evaluation Program From the Beginning. Kelly R. Brantley Health Assistance Partnership 13th Annual SHIP Director's Conference. Why Program Evaluation?. Why Program Evaluation?. Basic SHIP Grant for FY 2007

claude
Download Presentation

The Big Bang: Creating an Evaluation Program From the Beginning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Big Bang:Creating an Evaluation Program From the Beginning Kelly R. Brantley Health Assistance Partnership 13th Annual SHIP Director's Conference

  2. Why Program Evaluation?

  3. Why Program Evaluation? • Basic SHIP Grant for FY 2007 • Provide locally-based individual counseling services to a greater number of individual beneficiaries unable to access other channels of information, • Increase targeted outreach to diverse and hard-to-reach populations, • Increase and enhance the counselor workforces • Increase local and State participation in CMS education and communication activities

  4. Informs program planning and delivery Involves stakeholders Supports learning Respects community Test a theory Tell a story Be accountable Inform the field Support fundraising Understand and improve program Why Program Evaluation?

  5. What Does Program Evaluation Look Like?

  6. What Does Program Evaluation Look Like? • Forms and Reports: CCFs, PAMs, Resource Reports, State database • Surveys or Interviews: Client satisfaction, training, outreach, presentations • Performance Review: Top-down and bottom-up strategies • Effectiveness or cost-effectiveness studies

  7. Evaluation Myths • It is too complex. • We did it once before. • We don’t have enough resources. • We don’t know if we will do it right. • I already know what we need to do.

  8. What Do You Do With Your Numbers?

  9. What Do You Do With Your Numbers? • Tabulate • Report to CMS • Draw trend lines • Decide where to focus • Ask and answer questions • Develop agenda items

  10. Implementation Evaluation • Answers who, what, when, where, why • Also called process evaluation • In SHIPs, implementation evaluation is most common

  11. Implementation Evaluation • An example: Evaluate a presentation to the public • Who was the audience? • How many people were there? • When was the presentation? • How long was the presentation? • What were the topics? • What did people think of the presentation?

  12. Outcomes Evaluation • Measures changes in knowledge, skills, attitudes, behavior, motivation, decisions, policies, and conditions • Often asks a vague, immeasurable question • So must use an indicator to capture a value

  13. Outcomes Evaluation • An example: Evaluate a counselor training session • Do counselors understand the topic sufficiently to provide needed information to clients? • Indicator: Eighty percent of trained counselors should pass a post-test with a score of 80% or higher.

  14. Components of Outcomes Evaluation • Inputs • Activities • Outputs • Outcomes • Outcome targets • Outcome indicators

  15. Planning Evaluation • Answer basic questions: • What do you want to be able to decide from your results? • Who is the audience for the results? • What information should be gathered? • When do you need this information? • Where and how do you get the information? • What resources are available to get, analyze, and report the information?

  16. Resources • Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources:http://www.managementhelp.org/evaluatn/outcomes.htm • Evaluation Activities in Organizations:http://www.managementhelp.org/evaluatn/evaluatn.htm • Volunteer Program Evaluation:http://www.energizeinc.com/art/subj/prog.html • Innovation Network:http://www.innonet.org/ • Outcome Measurement Resource Network:http://national.unitedway.org/outcomes/library/ndpaper.cfm

  17. Questions?

  18. Thank You • Kelly R. Brantley • Senior Education and Training Associate • Health Assistance Partnership • Contact Information • shiphelp@hapnetwork.org • (202) 737-6340

More Related