Planning Outcomes-Based Outreach Programs

1. Planning Outcomes-Based Outreach Programs National Network of Libraries of Medicine Evaluation Office

2. You got to be very careful if you don’t know where you’re going because you might not get there-- Yogi Berra You got to be very careful if you don’t know where you’re going because you might not get there -- Yogi Berra

3. Why talk about outcomes in the planning phase? • What gets measured gets done • If you can’t recognize failure, you can’t correct it • If you cannot see success, you can’t reward it (and may be rewarding failure) • If you do not measure results, you cannot see success • If you can demonstrate results, you can win public support From Reinventing Government

4. Begin with the end in mind • – Stephen CoveyThe 7 Habits of Highly Effective People

5. A plan helps you move in the right direction. Evaluation keeps your eyes on the road

6. By the end of this workshop you will know how to • Use logic models to plan programs around desired results (outcomes) • Shape objectives from outcomes • Develop evaluation plans based on logic models

7. The Logic Model Creating the map

8. Begin with the results you’re seeking

9. Figure out what you have to do

10. Then identify what you need

11. Activities are more under your control than outcomes

12. Here are examples of the items for each column

13. Goal: Have a good summer vacation

14. Goal: Have a good summer vacation

15. Think of achieving outcomes in stages

16. Vacation: Output versus Outcome

17. Library Project: Output Focus

18. Library Project: Outcome Focus

19. There are two general categories of outcomes

20. Examples of system-level change

21. People change in stages

22. Kirkpatrick’s Model of Program Evaluation is popular for evaluating training programs

23. Kirkpatrick’s Model can be useful in completing the Logic Model

24. Last step: Get Feedback from stakeholders and key informants

25. How could the logic model help plan proposals?

26. Use the logic model to write final reports

27. Making outcomes measurable: Writingobjectives

28. Build objectives around your outcomes

29. Indicators are the observable signs of the outcome Outcome Indicator (measurable) Parents will report they were able to use MedlinePlus to locate information on at least one health topic that was important to them Parents will increase their ability to find online health information

30. The target sets the criteria for success Indicator (measurable) Target (criterion for success) 70% of participants Parents will report being able to use MedlinePlus to locate information on at least one health topic that was important to them

31. Time frame tells you when you are going to check on the indicator

32. Now you can write the outcome objective for an outcome Outcome Outcome Objective By the end of the training session, 70% of parents will report being able to use MedlinePlus to locate information on at least one health topic that was important to them Parents will increase their ability to find online health information

33. Example Objective: Awareness Outcome Outcome Objective By the May 2015, All 8 regional medical libraries will have re-posted OERC blog articles in their Newsletters or blog The OERC will improve access to information about emerging evaluation trends among NN/LM members

34. Objective Example: Impact Outcome Outcome Objective One month after training, 60% of clinicians will report providing at least one patient with Information about MedlinePlus Clinicians will recommend health information to their patients

35. Discussion: What indicators could you use? Increased use of health resources Indicator: ____________________ More access to health resources Indicator: ____________________ More cost effectiveness Indicator: ____________________ Increased efficiency Indicator: ____________________

36. Webinar 4: Adding Evaluation to Your Plan

37. What your stakeholders want to know • Did you get there? • Did you do everything you planned? • How Did you get there? • Did you do everything you planned? • What unexpected things happened? • Would you go there again? • What advice do you have for others? • What would you do differently next time? • What nexpected things happened? • Would you go there again? • What advice do you have for others? • What would you do differently next time?

38. Focus on “Rapid” methods • Short surveys • Site visits • Observations • Key informant and focus group interviews • Existing data review

39. Use mixed methods Quantitative methods for numbers Qualitative methods for text or images

40. Source and Method Where we find data How we collect data

41. Process evaluation: Keeping your eyes on the road

42. Process assessment is developed for planned activities

43. Documenting process

44. How to describe progress

45. Add check-ins with key players

46. Comments and feedback allow you to assess quality of your program

47. Outcomes Assessment: Did you make it and was it worth it?

48. Outcomes assessment is guided by the “intended results’ columns

49. How do you evaluate your outcome objective? By the end of the training session, 70% of parents will report being able to use MedlinePlus to locate information on at least one health topic that was important to them

50. Identify Source, Method, Timing By the May 2015, All 8 regional medical libraries will have re-posted OERC blog articles in their newsletters or blog