Planning and Evaluating Health Information Outreach Projects

1. Planning and Evaluating Health Information Outreach Projects Susan Barnes, MLS Cynthia Olney, PHD Outreach Evaluation Resource Center

2. Introduction • Getting Started with Community-Based Outreach(Booklet 1) • Including Evaluation in Outreach Project Planning (Booklet 2) • Collecting and Analyzing Evaluation Data (Booklet 3)

3. Workshop Objectives By the end of the workshop, you should be able to: • Create a plan for finding outreach partners • Develop strategies for conducting community assessment • Design an outcomes-based project using the logic model as a tool

4. Workshop Objectives By the end of the workshop, you should be able to: • Write measurable outcomes • Develop strategies for outcome, pre-program, and process evaluation • Understand basics of collecting and working with data

5. Evaluation Standards • Utility • Feasibility • Propriety • Accuracy From Preface (page ii of each booklet)

6. Community-Based Outreach • Outreach means health information programs go to the target population • Primary goal is capacity building (individuals; organizations) • Partnerships are formed among organizations who can pool assets and resources

7. Find Partners for Health Information Outreach Networking • Health fairs • Local health or social service coalitions • State associations • Other examples? See Booklet 1, Appendix 1 (page 7)

8. Partnerships • Organizational readiness • Mutual agreement • Write it down! • Revisit expectations periodically

9. Community Assessment From Booklet 1, Step Two (page 3-4)

10. Community Assessment Diffusion of Innovation • Innovators (2.5%) • Early adopters (13.5%) • Opinion leaders

11. Primary Contacts - Innovators • Enthusiasm • Need access to health information • Have values, habits, and behaviors consistent with innovation • Can “practice” with the innovation • Will see tangible results

12. Community Assessment Stages of Change (Individuals) • Pre-contemplation • Contemplation • Preparation • Action • Maintenance

13. Community Assessment • Primary contacts • Resources and assets • Health information access • Computer literacy • Computer access • Influences and challenges

14. Developing an Outcomes-Based Project From Booklet 2, Step One (page 2-6)

15. Logic Models • What gets measured gets done • If you do not measure results, you cannot tell success from failure • If you cannot see success, you can’t reward it • If you cannot reward success, you probably are rewarding failure • Ellen Taylor-Powell, UNW-EX

16. Logic Models You got to be very careful if you don’t know where you’re going because you might not get there -- Yogi Berra

17. Logic Models Activities /Outputs Resources Outcomes If we get these resources… Conduct these activities and deliver these services/ products We will accomplish these outcomes

18. Logic Models Activities/ Outputs Resources Outcomes Planned Activities Intended Results

19. Outcome Driven Activity Driven Outcome Driven Increase the number of resources that parish nurses use to get consumer health information Train 50 parish nurses on MedlinePlus and other NLM resources

20. Results Behavior Learning Reaction Kirkpatrick Model

21. Individual-Level Outcomes • Knowledge • Affective and Attitude • Skills • Improved ability to manage health care

22. Community-Level Outcomes • Environmental • Social

23. Resources Activities /Inputs Outcomes Assumptions External factors Logic Model Logic Models

24. Exercise See Booklet 2, Step One (page 3)

25. Logic Models - Discussion How can logic models be used to build and sustain collaborations and partnerships?

26. Gone to Lunch See you at 1 p.m.

27. Building Objectives From Booklet 2, Step 2 (pages 7-10)

28. Outcome Indicator Target Building Outcome Objectives Time Frame

29. Identifying Indicators Parents will find useful online resources for researching health concerns related to their children Indicator: Parents’ rating of the usefulness of the online resources on a training session evaluation form

30. Identifying Indicators Outcome 1 Parents will have increased ability to locate children’s health information Outcome 2 Parents will identify ways to get help from local libraries in locating health information

31. Identifying Indicators Outcome 3 After training, parents will use online resources to research health concerns related to their children Outcome 4 Public librarians will report increases in health reference questions from either the training participants or parents referred by training participants or CBO staff

32. Writing Outcome Objectives Parents will find useful online resources for researching health concerns related to their children Target: 70% will rate resources as useful Time frame: By the end of the training session

33. Writing Objectives Outcome Parents will find useful online resources for researching health concerns related to their children Objective By the end of the training session, 70% of participants will rate the online resources as useful on the training session evaluation form

34. Writing Outcome Objectives Indicator (measurable) Participants will rate the online resources as useful on the training session evaluation form Target (criterion for success) 70% positive ratings from participants Timeline By the end of the training session

35. Outcome Objectives Exercise

36. Planning Pre-Program and Process Assessment From Booklet 2, Step Three (page 10-11) and Booklet 2, Step Four (page 11-12)

37. Pre-Project Assessment Plan Pre-Project Assessment • Review community assessment information • Review your logic model • Take an inventory of your resources • Conduct an audience analysis (Stages of Change)

38. Process Assessment Plan Process Assessment • How closely has the project followed the project plan (look at activities and outputs) • How well has the project been implemented (participant feedback) • What barriers need to be addressed (implementer feedback

39. Designing Evaluation Methods From Booklet 3, Introduction (pages 1-2)

40. Conducting Evaluation Step 1 Design your method Step 2 Collect Data Step 3 Summarize/Analyze Step 4 Assess Validity

41. What are your evaluation questions? How many? How much? What percentage? How often? What is the average amount? What worked best? What did not work well…? What do the numbers mean? How was the project useful…? What factors influenced success or failure? Choose Quantitative Methods Choose Qualitative Methods

42. Designing a Survey From Booklet 3, Quantitative Methods, Steps One and Two (pages 5-12)

43. Designing a survey • Identify evaluation questions • Decide on sampling strategy • Write survey questions that answer evaluation questions

44. Planning a Survey Exercise

45. Planning an Interview From Booklet 3, Qualitative Methods, Steps One and Two (pages 21-24)

46. Planning Interviews • Write evaluation questions • Determine sampling strategy • Write interview question guide

47. Planning an Interview Exercise

48. Workshop Evaluation

49. Workshop Objectives By the end of the workshop, you should be able to: • Create a plan for finding outreach partners • Develop strategies for conducting community assessment • Design an outcomes-based project using the logic model as a tool

50. Workshop Objectives By the end of the workshop, you should be able to: • Write measurable outcomes • Develop strategies for outcome, pre-program, and process evaluation • Understand basics of collecting and working with data