Building a Culture of Collaborative Inquiry to Improve Adolescent Literacy

1. Building a Culture of Collaborative Inquiry to Improve Adolescent Literacy Day 2, Logic ModelingJune 5, 2014Nashville, TN

2. Today’s presenters • Zoe Barley, Ph.D. Event facilitator, REL Appalachia, and Zbarley Consulting • Stephanie Wilkerson, Ph.D. Alliance lead, REL Appalachia, and Magnolia Consulting

3. Yesterday’s workshop ... • Identified: • Key root causes of barriers to effective data use. • Interventions currently under way in the district. • Gaps between root causes and interventions. • Developed long-term outcomes for the district, school, and classroom in resolving root causes. • Identified needs for professional learning and resources in order to improve data use.

4. The day ahead • Learn about logic models – purposes and creation. • Create an initial MNPS alliance logic model: • Outcomes refined and time frame projected. • Strategies/activities to accomplish. • Aligned to root causes.

5. What is a “logic model”? Many people say a logic model is a roadmap.

6. A logic model is … • A depiction of planned work showing: • What the work will do. • What the work will accomplish.

7. Complex logic model Source: State University of New York, 2011

8. Source: University of Wisconsin-Extension, 2003 A linear model

9. The basis of the “logic” • A series of “if-then” relationships that, if implemented as intended, lead to the desired outcomes.

10. Theory of change • A theory of change is a description of how and why a set of activities – be they part of a highly focused program or a comprehensive initiative – are expected to lead to early, intermediate, and long-term outcomes over a specified period.

11. An example of a theory of change Source: Center for Advancement of Informal Science Education , n.d.

12. Programs fail for 3 reasons • Theory of change failure: Program conceptualization and design cannot address the problem or cannot generate desired outcomes, no matter how well implemented. • Translation failure: Theory is sound, but translation into practice or action is not. • Implementation failure: Failure to properly implement programs, strategies, or activities. Adapted from Peter Rossi. Source: Shadish, Cook, & Leviton, 1991

13. How is a logic model useful? • Helps differentiate “what we do” from “results”. • Clarifies the results; keeps focus on priorities. • Represents the big picture; blueprint. • Provides common language. • Identifies gaps or inconsistencies. • Fosters ownership and accountability. • Provides a foundation for evaluation.

14. Simple logic model

15. Target population • The audience you will affect through your activities: • Students. • Teachers. • Parents. • Leaders. • Community.

16. Inputs • The resources you invest in a program: • Funding. • In-kind contributions. • Staff. • Time. • Materials and equipment. • Partnerships. • Planned activities/strategies. • The state and local interventions already in place.

17. Activities • What do you do to reach and affect the target audience: • Changes to team agendas to incorporate collaborative inquiry. • Collection of new data. • In-house professional development on data visualization. • How you address a clearly defined need (root cause) in the target population or community.

18. Outputs – Short-term/intermediate outcomes • First steps toward the desired impact: • Time for teachers to collaborate. • Faster data turnaround. • New ability to visually display data.

19. Outputs vs. outcomes • Example: Teachers meeting for collaborative inquiry is an output. Change in instructional practice based on the learning from the inquiry is an outcome. Outcomes – “Not how many worms the bird feeds its young, but how well the fledgling flies.” - United Way of America

20. Outcomes • Short-term • Intermediate • Long-term

21. Hierarchy of outcomes Source: Adapted from Rockwell & Bennett, 1995

22. Recap: The logic of the logic model • The long-term outcomes are the impact we desire. • The intermediate outcomes will lead to the long-term outcomes: • If we accomplish them, then … • The short-term outcomes will lead to the intermediate outcomes: • If we accomplish them, then …

23. Recap: The logic of the logic model • The activities we have planned and the target audiences we have engaged will lead to the short-term outcomes. • The activities address the root causes that create barriers to effective data use, hindering the improved adolescent literacy we seek.

24. Next steps • Where are you going? • How will you get there? • What will show that you’ve arrived? Source: University of Wisconsin-Extension, 2003

25. Today’s focus • If we implement a collaborative inquiry approach to data use, what classroom-, school-, and district-level outcomes do we intend to achieve? • Articulating outcomes for: • How we work together. • The way in which we use data. • The knowledge and skills for each.

26. Overview of logic modeling steps • Step 1: Review root causes and long-term preferred outcomes. • Step 2: Develop short-term and intermediate outcomes. • Step 3: Identify strategies and activities. • Step 4: Review as a whole group.

27. Three groups: Classroom, school, district • Classroom outcomes (teachers) • What will teachers know and be able to do in their literacy instruction as a result of participating in collaborative inquiry? • School outcomes (coaches, principals/assistant) • What will coaches and administrators know and be able to do in their schools as a result of participating in and supporting a collaborative inquiry process for data use in literacy instruction? • District outcomes (district staff, directors, executives) • What will district staff know and be able to do in MNPS as a result of fostering a districtwide culture of collaborative inquiry for data use in literacy instruction?

28. Logic Modeling Step 1 • Review root causes and long-term preferred outcomes (from yesterday). • Facilitators have made suggestions. • What additional suggestions does the group have?

29. Logic Modeling Step 2 • Develop short-term and intermediate outcomes. Given a long-term outcome … For example: Literacy instruction is based on findings from collaborative inquiry. … what would be an intermediate step toward obtaining it? For example: Teams of literacy teachers practice with collaborative inquiry processes.

30. Logic Modeling Step 2 What would be a short-term step toward the intermediate outcome? For example: Literacy teachers will know how to engage in collaborative inquiries.

31. Logic Modeling Step 3 • Identify strategies and activities. What current work (activity of strategy) do we need to modify, or what new work should we consider, to attain the short-term outcomes? For example: Professional learning on collaborative inquiry is provided to middle school teachers.

32. Is the model logical? • Is our progression from activity to shorter-term outcomes logical?

33. Do the “If …Then’s” work? Source: Center for Advancement of Informal Science Education , n.d.

34. Logic Modeling Step 4 • Review as a whole group. • How do the outcomes support addressing the root causes identified yesterday? • Is there continuity across levels? Are there gaps? How can we address these? • How could we use these logic models to guide our collaborative inquiry work and promote collaborative inquiry in middle schools and throughout the district?

35. Next steps • Ongoing learning based on Dr. Lipton’s presentation and in preparation for Day 2 of her workshop on July 15, 2014. • Distribution of additional resources, as identified over the course of the technical assistance. • Digital transfer and clean-up of logic model development from today. • Preparation for follow-on session on innovation configurations on July 16, 2014.

36. REL Appalachia Stakeholder Feedback Survey • Did we meet our goals and objectives for the fishbone process and logic modeling activities? • Do you have other support or information needs? • Anonymous. • 5-10 minutes.

37. References Anderson, A. (2000, November). Using theory of change in program planning and evaluation. Paper presented at the annual meeting of the American Evaluation Association, Honolulu, HI. Center for Advancement of Informal Science Education. (n.d.). Principal investigator’s guide: Managing evaluation in informal STEM education projects . Retrieved March 13, 2014 from http://informalscience.org/evaluation/evaluation-resources/pi-guide#introduction. Office of Financial Management, State of Washington. (2008). Enterprise risk management. Retrieved March 21, 2014 from http://performance.wa.gov/GE/GE111908/RiskManagementWorkerSafety/ERMMaturity/Documents/ERM%20Logic%20Model.jpg. Rockwell, K., & Bennett, C. (2004). Targeting outcomes of programs: A hierarchy for targeting outcomes and evaluating their achievement. Faculty Publications: Agricultural Leadership, Education & Community Department. Retrieved October 20, 2013 from http://digitalcommons.unl.edu/aglecfacpub/48. Shadish, W., Cook, T., & Leviton, L. (1991). Foundations of program evaluation: Theories of practice. Thousand Oaks, CA: Sage Publications. State University of New York. (2011). Catchup and complete enhanced blended learning initiative.Retrieved March 21, 2014 from http://wiki.sln.suny.edu/display/SLNNGLC/revised-updated+logic+model. University of Wisconsin-Extension. (2003, February). Enhancing program performance with logic models. Retrieved October 20, 2013 from http://www.uwex.edu/ces/pdande/evaluation/pdf/lmcourseall.pdf.