Community Systems Group, LLC 2013

1. Sustainability & Evaluation: Tuning Up for Coalition Success Community Systems Group, LLC 2013 Helping you create healthier communities

2. Brenda Bone, MA Managing Director, Community Evaluation Cindy Pharis, MS Community Research & Evaluation Specialist Helping you create healthier communities

3. Learning Objectives • Note the five questions of evaluation; Use self-reflection to answer how it applies to your own coalition. • Identify the four major crossroads at which evaluation and sustainability meet. • Practice key elements of sustainability planning. Helping you create healthier communities

4. Sustainability & Evaluation: • Tuning Up Coalition Success What brings you here? Sustainability is…. Helping you create healthier communities

5. Evaluation Evaluation is the word used to describe how a coalition gathers information and uses that information to diagnose past actions as well as carefully plan for the future. • Sustainability Sustainability is the community’s on-going capacity and resolve to work together to establish, advance, and maintain effective strategies that continuously improve health and quality of life for all. - CDC’s Healthy Communities Program, “A Sustainability Planning Guide for Healthy Communities” www.cdc.gov/healthycommunitiesprogram/pdf/sustainability_guide.pdf Helping create a healthier communities Helping you create healthier communities

6. Evaluation is necessary for Sustainability • Strengthen collaboration • Demonstrate worth and importance • Stakeholders are committed • Flexibility & Improvement • Integrate multiple issues • Throw the tent around influences • Contribute • Collaborative • Worth and Importance • Stakeholder Questions • Flexibility & Improvement • Integrate all the data • Account for Influences • Contribution Helping you create healthier communities

7. The Strategic Prevention Framework

8. SPF Re-visioned

9. Evaluate to Sustain • If you haven’t written it down, it doesn’t exist. • Having the most data does not mean you win. • Having no data does mean you lose. Helping you create healthier communities

10. The FIVE QUESTIONS of Evaluation • External Audiences: Funders, Supporters, Community • Internal Audiences: Staff, Volunteers, Managers, Board, Partners • Improving Program/Effort Accountability • Communication, Project Coordination • Short- and Long-Term Results • Process: how we do it, what we did • Short-term outcomes: Results • Long-term outcomes: Impact • Existing Data – create a protocol for gathering this information. • Created Data – what instruments do we use to create data? • Decision making cycles (when); Audience (what); Method (how) • Balance a visual with a written presentation. Seek feedback. Helping you create healthier communities

11. Evaluation Plan Helping you create healthier communities

12. Four Places Evaluation and Sustainability Meet Helping you create healthier communities

13. Define Your Community • Definition: A “group” drawn together by one or more commonalities, including geographic, professional, socioeconomic or cultural. • Sustainability • To remain relevant, leaders must work with multiple definitions of community within the same community. • An open definition liberates communities to recognize their natural features, assets, and interdependencies and to be of value to citizens on multiple issues addressed on multiple levels. Helping you create healthier communities

14. Are Your Anchors in Place?: Vision • Definition: Describes the clear and inspirational long-term desired change • resulting from a coalition’s work. • What is this to do with sustainability? • Deep vision takes time and resources. 5 – 10 years to implement. • A shallow vision, or one not owned and committed to, is no vision at all. • A little vision is an oxymoron. • A vision by its nature in expansive. • Visions are dynamically incomplete. • A few evaluation questions: • Does anyone know it? • Does anyone use it? • Does it appear anywhere? Helping you create healthier communities

15. Are Your Anchors in Place?: Mission • Definition: What you are going to do to create your vision that focuses on your philosophy, goal, ambition, mores. • Sustainability • Flexible and adaptive to changing environments and times. • No substitute for local articulation. • A few evaluation questions: • Does anyone know it? • Does anyone use it? • Where can we find it? Helping you create healthier communities

16. Are Your Anchors in Place?: Theory of Change • Definition: Statement of your belief that what you are proposing to do will have the effect you intend it to have. When we (____) and focus on (_____), we will create or contribute to (_____). Examples: By utilizing community problem solving techniques and evidence based strategies we will contribute to creating a healthier community. By engaging our schools in setting strong standards for our youth, focused especially on strengthening our entire student assistance, discipline and referral processes, and all the community sectors we need to provide these services, we will reduce substance abuse and increase student retention. Helping you create healthier communities

17. Are Your Anchors in Place?: Theory of Change • Sustainability • Fragmented and disconnected problem – solving slow the process and waste your resources. • Fragmentation threatens long-term effectiveness. • Look to collaboration between collaboratives. • A few evaluation questions: • Do the vision, mission and theory of change align? • Who is part of your community-building process? • Is your coalition duplicating efforts or strategies? Helping you create healthier communities

18. Are Your Anchors in Place?: Goals • Sustainability: • A problem statement and a goal are frequently used interchangeably. • Goals should address one thing at time and be specific enough to be actionable • Goals should be broad enough that interventions may be adjusted (as a result of evaluation) without altering the goal. • Goals should have the ability to be worked on by multiple people or sectors. • Evaluation: • Does goal have a repeatable baseline measure? • Is more than one data source used to verify the goal? • Does the data being used relate to the goal statement? • Is the data useful? Helping you create healthier communities

19. How Is Your Project Framed?: Root Causes • Evaluative Questions: • Do you have data that relates to each root cause? • Can you repeat the measure? • How do you know that the root cause you are describing relates to the goal? • Is this comprehensive? Have you prioritized the list? • Who decided these are your root causes? • Do you have agreement in your coalition? • Sustainability: • Root causes are grounded in predictive theory: There’s never a cure at this level; the coalition must constantly address these. • Since you’ve prioritized, things have fallen off the map. These can be looked at in the future. Helping you create healthier communities

20. Is Your Project Framed?: Local Conditions • Evaluation Questions: • A well-framed local condition almost writes its interventions. • Qualitative data begins to show up here – especially about KAFB. • Can you repeat any of these measures? • Is the list comprehensive? • Is it prioritized? • Do you have consensus about them? • Sustainability: • Local conditions where the action takes place, and where you get the beginnings of your ability to demonstrate success – or what barriers you encountered. • Local conditions are rarely “fixed” permanently; local conditions will shift – sometimes because of your effort, and sometimes because the population just changes. Helping you create healthier communities

21. Do You Have the Guts of It?: Interventions • Sustainability: • Demonstrates your decision-making process in selection of a particular intervention. • Action plans demonstrate the implementation process (good and bad) of your selected interventions. They show what you are doing and how it is going. • Allows for prediction of future needs (or not) for a particular intervention, and allows demonstration of why additional interventions may be needed. • Evaluation Questions: • Is there evidence of use of the 7 Strategies? • Are the interventions evidence-based? • Are local readiness, timing, dose and capacity being taken into account in selection of the interventions? Helping you create healthier communities

22. Do You Have the Guts of It?: Capacity Building • Sustainability: • Efforts focused on building civic infrastructure and social capital? • Improve communication and information sharing. • Improve inter-sectoral and inter-group relations. • Work toward true collaboration. • Set up win-win versus win-lose mentality. • Evaluation: • How are the members involved in implementation? • Does each member understand and value their role in the whole picture? • Does the coalition structure create efficiency and effectiveness? • Is the coalition structured to create the space for continuous improvement? Helping you create healthier communities

23. Acknowledging Common Challenges • Community is a complex system • Problems are multi-determined • Problems are syndemic • Agents of change are also targets of change • Changing and evolving interventions • Multi-level • Interactive & synergistic effects Helping you create healthier communities

24. Sustainability Plan Helping you create healthier communities

25. Interventions, Sustainability and Evaluation Use the names that come from the list of Seven Strategies… and Capacity Building Contain a verb. There are frequently many action steps to completing one strategy. People Places Materials Time Can you put a number to any of this? Who will do the work? NAMES As specific a date as possible. At the least, a month.

26. Community change takes time and patience Healthy communities celebrate, have fun, and pay attention to the beauty life offers along the way. This process lasts a life time. - “Sustaining Community Based Initiatives” module of “Developing Community Capacity.” From The W.K. Kellogg Foundation in partnership with the Healthcare Forum. community-wealth.org/_pdfs/tools/cdcs/tool-kellogg-cmty-cap.pdf

27. Cindy Pharis, MS Research & Evaluation Specialist cindy@communitysystemsgroup.com 314-287-5164 Brenda Bone, MA Managing Director, Community Evaluation brenda@communitysystemsgroup.com 314-287-5161 Helping you create healthier communities