Evaluating Advocacy: A Model for Public Policy Initiatives

1. Evaluating Advocacy: A Model for Public Policy Initiatives

2. National nonprofit organization Committed to evaluation as a tool for empowerment Work with nonprofits at the grassroots level Build evaluation capacity of nonprofits and funders so they can better serve their communities Practice a participatory approach Innovation Network (202) 728-0727 1625 K St. NW www.innonet.org Washington, DC 20006 info@innonet.org Who We Are: Innovation Network

3. Overview/Agenda • Discuss challenges inherent to advocacy evaluation • Review traditional evaluation models and their fit for evaluating advocacy • Share lessons learned from our work • Evaluation planning • Data collection strategies • Feedback loops and reflection • Shift in emphasis

4. A Case Study Coalition for Comprehensive Immigration Reform (CCIR) • Evaluation funded by The Atlantic Philanthropies • Began the evaluation in Fall 2005

5. THECHALLENGES

6. The Challenges I continue to find my greatest pleasure, and so my reward, in the work that precedes what the world calls success. -- Thomas Edison

7. The Challenges Success is to be measured not so much by the position that one has reached in life as by the obstacles which he has overcome while trying to succeed. -- Booker T. Washington

8. The Challenges Ability is nothing without opportunity. -- Napoleon Bonaparte

9. The Challenges If you don't risk anything, you risk even more. -- Erica Jong

10. TRADITIONAL MODELS

11. Continuous Learning Cycle Logic Model Analysis, Reflection & Improvement Evaluation Planning Data Collection

12. The Logic Model

13. Chain of Outcomes Residents improve basic job skills and identify jobs & career paths of interest KNOWLEDGE/ ATTITUDE Participants use post-training services, e.g. job placement, child care, transportation BEHAVIOR Increased number of community residents obtain and retain jobs that support families CONDITION Closer in TimeEasier to MeasureMore Attributable to Project More Distant in Time Harder to Measure Less Attributable to Project

14. LESSONS LEARNED

15. Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

16. Locate issue on the policymaking continuum Identify type of policy change Understand role of the group/network in policymaking Clarify dimensions of success relevant to stakeholders Develop chain of outcomes, leading to the intended change Identify indicators & measurement systems Identify most effective strategies Prioritize 8-Step Advocacy Evaluation Plan

17. Spheres of Influence

18. Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

19. Data Collection Strategies • Mixed-method approach • Deeper focus on “real-time” data • Observations/monitoring • Frequent opportunities for data collection • De-brief protocol • Opportunity windows/intense periods • Capturing multiple perspectives

20. Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

21. Reporting and Feedback Loops • Frequent, informal reporting • Learning briefs; research • Approximate “real-time” • Transparency/inclusion • Face-to-face

22. SHIFT IN EMPHASIS

23. Before… Outcomes Outputs

24. Now… Outputs Outcomes

25. Finding Balance… Outputs Outcomes

26. Advocacy Evaluation Project • Online resources • Indicators for advocacy • Piloting tools • Advisory committee www.innonet.org/advocacy

27. Any Questions? www.innonet.org Jennifer Bagnell Stuart jabstuart@innonet.org Ehren D. Reed ereed@innonet.org

28. Thanks for Your Participation! Measure results. Make informed decisions. Create lasting change. Innovation Network, Inc. 1625 K St. NW, 11th Floor Washington, DC 20006 (202) 728-0727 www.innonet.org