Theories of Change and Logic Modeling: Matching Means and Ends

1. Theories of Change and Logic Modeling:Matching Means and Ends Effective Professional Development Resulting in Long-Term Student Achievement NCLB/ITQ Symposium October 10, 2008

2. IBHE Model for Evaluation

3. Theory of Action • Universal Theory of Action: • Collaborative partnerships are the mechanism for action • Project Theories of Action: how each intervention is delivered • train-the-trainers model • on-line learning model • direct instruction model • cooperative learning model • workshop series model • mentor/mentee model • The Theory of Action describes HOW a project is delivered, usually based on the professional development literature

4. Limitations of Theories of Action • They describe HOW a project is delivered, not WHY it will work or what EFFECTS it will have. • They are therefore evaluated by outputs, not by outcomes/effects. • Output evaluation is necessary, but not sufficient, to show value for the project.

5. Theory of Change: Three Elements

6. Element One: The Change Framework • A backward mapping process that begins with Outcomes to plan, implement, and continuously develop and improve programs

7. Element Two: Program Interventions • Designing interventions or activities intended to create desired results, and linking them systematically

8. Element Three: Program Assumptions • Carefully articulating assumptions about how change will occur and testing the assumptions systematically through evaluation

9. Theory of Change: Background • Evaluation is challenging because program assumptions are not well-articulated. • The more complex the problem, the more difficult it is to be clear about how things will change as a result of the program. • By designing a Pathway of Change, program developers/ implementers/ evaluators, can strengthen the case that an intervention caused a result.

10. Theory of Change: Process • Backward mapping from long, short, and mid-range goals or outcomes laid out in sequence • Link outcomes to program elements • Plan evaluation from the outset to track achievement of the intended outcomes • Use evaluation results for improvement and accountability

11. Broad Benefits of Theories of Change • “Stakeholders value theories of change as part of program planning and evaluation because they create a commonly understood vision of the long-term goals, how they will be reached, and what will be used to measure progress along the way.” • ActKnowledge/The Aspen Roundtable • www.theoryofchange.org

12. Theory of Change:Project Benefits • Makes explicit the basis for the project intervention • Consolidates the research that undergirds the project • Provides an overarching framework for project planning, implementation and evaluation

13. Theory of Change and Logic Modeling A Logic Model is a graphic form of a Theory of Change • Takes the theory, which is selected on the basis of empirical evidence that it will work, and translates it into an operational plan. • Created by backward mapping.

14. The “IfThen” Connections • IF certain inputs are available… • THEN certain activities/interventions can take place; • IF these interventions occur… • THEN certain outputs will be seen; and • IF the interventions are effective… • THEN certain outcomes will be attained. • Finally, what we learn from the outputs and outcomes can be used to improve the intervention.

15. Building the Logic Model • Start with the Theory of Change: “Companies do better when their employees are healthy.” • Cite empirical evidence for this theory.

16. Building the Logic Model • Do the backward mapping • Ask and answer the “How?” questions Centers for Disease Control Healthier Worksite Initiative http://www.cdc.gov/nccdphp/dnpa/hwi/program_design/logic_model.htm

17. Building The Logic Model • Lay out the steps in order from inputs to outcomes • Answer the “Why?” questions. Centers for Disease Control Healthier Worksite Initiative http://www.cdc.gov/nccdphp/dnpa/hwi/program_design/logic_model.htm

18. Building The Logic Model • Create the model with appropriate connections INPUTS • Funding • Staff • Scientific Evidence

19. Building The Logic Model • Create the model with appropriate connections INPUTS INTERVENTION • Funding • Staff • Scientific Evidence Workplace Walkability Program

20. Building The Logic Model • Create the model with appropriate connections INPUTS INTERVENTION OUTPUT • Funding • Staff • Scientific Evidence Employees walk to meetings more often Workplace Walkability Program

21. Building The Logic Model • Create the model with appropriate connections INPUTS INTERVENTION OUTPUT OUTCOMES Employee level of physical activity and fitness increases • Funding • Staff • Scientific Evidence Employees walk to meetings more often Workplace Walkability Program Productivity Increases

22. Building The Logic Model • Create the model with appropriate connections • Identify your evaluation measures for outputs and outcomes INPUTS INTERVENTION OUTPUT OUTCOMES Employee level of physical activity and fitness increases • Funding • Staff • Scientific Evidence Employees walk to meetings more often Workplace Walkability Program Productivity Increases Employee Logs Employee Fitness Screenings Company Data

23. Project Benefits of Logic Models • They make a stronger case for project investment – funders understand more clearly what you intend to do • They reflect group process and understanding among project staff and project participants • They indicate where and how to evaluate program effects

24. We Now Find These Everywhere U.S. Department of Health & Human Services Idaho State Board of Education

25. We Now Find These Everywhere National Institute of Standards and Technology processoutcome.org Research Utilization Support & Help (RUSH) Canadian Department of Justice

26. NOT Recommended! Ohio Department of Education (ODE)