Building a Culture of Collaborative Inquiry to Improve Adolescent Literacy Day 1, Fishbone Process

1. Building a Culture of Collaborative Inquiry to Improve Adolescent LiteracyDay 1, Fishbone Process June 4, 2014 Nashville, TN

2. Today’s presenters Zoe Barley, Ph.D. Event Facilitator, REL Appalachia, and ZBarley Consulting Stephanie Wilkerson, Ph.D. Alliance Lead, REL Appalachia, and Magnolia Consulting

3. Yesterday’s workshop ... • Collaborative Learning Cycle. • Simulated data-use practices with local data. • Interactive conversations using qualitative and quantitative data. • Patterns and trends in literacy data displays. • Theories of causation and action promote thoughtful literacy improvement and planning.

4. MNPS Data Use Research Alliance Goals 2014‒2016: • Identify and implement best practices in data use that improve adolescent literacy instruction. • Identify and apply the tools, processes, and skills necessary to implement a collaborative inquiry approach for data use in adolescent literacy instruction.

5. MNPS Data Use Research Alliance Proposed activities for 2014: • Collaborative inquiry workshops with Dr. Laura Lipton. • Fishbone session to identify root causes of barriers to effective data use. • Logic modeling session to develop outcomes for effective data-use practices. • Innovation configuration session to develop a collaborative inquiry blueprint. • Survey of collaborative inquiry and data-use practices.* • Bridge event on five key steps for structuring conversations about data.* *Pending project approval by the Institute of Education Sciences

6. Outcomes for fishbone and logic model activities • Increase awareness of root causes. • Engage in a gap analysis to reveal misalignment among root causes, interventions, and intended outcomes. • Apply knowledge of data-use best practices to developing intended outcomes.

7. Outcomes for fishbone and logic model activities • Experience the use of: • Fishbone process for identifying root causes of problems confronting the district, such as low literacy achievement and insufficient data use. • Logic modeling to depict causal relationships among root causes, interventions, and intended outcomes related to data use in literacy instruction and professional learning.

8. Group norms for working together We will… • Listen and be receptive of diverse perspectives. • Balance participation so all voices are heard. • Actively participate; minimize distractions (i.e., cell phones). • Support the group processes. • Evaluate the group processes.

9. Introductions • Name. • Position. • School, department, or organization.

10. Introduction to the fishbone

11. What is “fishbone analysis”? • Kaoru Ishikawa in 1960s: “cause-and-effect analysis” • Diagram-based approach for thinking through all possible causes. • Thorough analysis; consensus on key issues to address. • “Bones”—categories of causes. • “Head”—effect or problem that results from the causes.

12. Also referred to as “root cause analysis” Source: ThinkReliability, 2011

13. Why is understanding the problem important? “You cannot solve a problem  from the same consciousness that created it.” —Albert Einstein

14. Why is understanding the problem important?

15. Sample fishbone: Clarifying reading strategy Source: Center for Data-Driven Reform in Education , 2011

16. Fishbone development process overview • Step 1: Identify the root causes of barriers to effective data use. • Step 2: Share root causes across groups. • Step 3: Identify current interventions at local and state levels. • Step 4: Observe alignment of interventions to root causes. • Step 5: Develop long-term outcomes you would expect if root causes are addressed. • Step 6: Review long-term outcomes with the whole group. • Step 7: Identify professional learning and resource needs.

17. Fishbone for low literacy achievement

18. Fishbone for barriers to effective data use

19. Problem: Barriers to effective data use • The things or conditions that limit teachers, coaches, and school and district leaders from drawing meaning from data to inform practice. “We want to help teachers use data, not be used by data.” —Jeffrey Wayman

20. Causes related to data use barriers: 5 domains

21. Domain: Collaborative inquiry in data use • Uncovering assumptions; making predictions. • Generating hypotheses or research questions. • Probing for causality between actions and outcomes. • Making inferences, drawing conclusions, calibrating data. • Transforming findings into actionable knowledge and multiple theories of solution. • Fostering a culture of valuing data use. • Engaging in high-performing group strategies. • Is the learning environment supportive of collaborative inquiry? • Do educators trust one another to engage in collaborative inquiry? • Does the educational culture reflect the attitudes, values, goals, norms of behavior, and practices that characterize the importance and powerthat data can bring to the decision-making process? • Is there the opportunity and time for collaborative inquiry to occur? • Is there capacity to engage in the Collaborative Learning Cycle?

22. Domain: Data collection • Access to different types of data (e.g., summative, formative, diagnostic, qualitative, quantitative). • Data quality – valid and reliable, free of errors. • Access to data sources. • Availability of data collection instruments. • Access to and collection of the right data to answer questions. • Timely collection of or access to data. • Given the things that influence student learning, what data can we access or collect to measure them? • Are there instruments available to collect the data needed, or do they need to be created? • Are we collecting the right data from the right data sources to test hypothesesand evaluate outcomes?

23. Domain: Data analysis and interpretation • Organizing and synthesizing data. • Observing and explaining patterns and trends. • Using visual displays. • Aggregating and disaggregating data based on research questions. • Describing data statistically. • Involving stakeholders in sense-making process. • Identifying or confirming hypotheses. • Linking performance to learning objectives, learning needs, and causal factors. • Linking data to instruction. • Do we have a process for synthesizing information across multiple data sources? • Do we disaggregate data to reveal those students needing additional support? • Do we engage in collaborative processes for interpreting results? Do we seek alternative explanations for the results obtained? • Do we address our original research questions or hypotheses through our interpretations?

24. Domain: Data use preparedness • Do teachers, principals, and other leaders have the skills and knowledge to effectively use data? • Do data coaches and instructional coaches have the skills and knowledge to effectively use data? • How might teachers’ and leaders’ perceptions and attitudes about data influence their schools’ culture of using data? • Inquiry processes. • General data-use skills. • Understanding of data quality. • Understanding of data properties. • Data-use procedural skills. • Transformation of data to useful information for decision making. • Transformation of data to implementation. Source: Mandinach & Gummer, 2012

25. Domain: Data system infrastructure • What does the data system require to maximize data use? • How much time do teachers need to effectively use data? Leaders and coaches? • What are the fiscal resources needed for a district to effectively use data? • Do leaders have an explicit vision for data use? • Development, execution, and supervision of plans, policies, programs, and practices that control, protect, deliver, and enhance the value of data and information assets. • Ensures the accessibility of data, the quality of data, motivation for use of data, timeliness of data, staff capacity and support, curriculum pacing to allow for data use, time for data use, and organizational culture and leadership.

26. Three small groups

27. Fishbone for barriers to effective data use

28. Fishbone Process Step 1: Identify root causes

29. Fishbone Process Step 2: Share with the whole group • Each small group presents the root causes it has identified to the whole group. • The whole group asks questions for clarity and makes suggestions. • Small groups note similarities and differences in root causes identified.

30. Fishbone Process Step 3: Identify interventions

31. Fishbone Process Step 4: Observe alignment

32. Fishbone Process Step 5: Develop long-term outcomes • Consider classroom, school, and district outcomes, for example: • Teachers use data as an integral component of their literacy instruction. • Schools foster a culture of collaborative inquiry for data use in literacy instruction. • MNPS provides teachers and leaders with a robust and responsive data infrastructure that supports a collaborative inquiry approach for data use in literacy instruction.

33. Fishbone Process Step 6: Review with the whole group • Groups share out on fishbones. • Note similarities and differences in: • Root causes. • Outcomes. • Cross-group reflections on similarities and differences: • Identify overarching root causes and outcomes.

34. Fishbone Process Step 7: Identify needs

35. Tomorrow’s focus: Seeing the forest for the trees • Root causes and long-term outcomes >>> logic models. • Long-term outcomes >>> intermediate and short-term outcomes. • Activities, strategies, and interventions aligned with outcomes.

36. Center for Data-Driven Reform in Education. (2011). Root-cause analysis: Clarifying. Retrieved February 12, 2014, from http://www.cddre.org/achievement/root.html. Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using student achievement data to support instructional decision making (NCEE 2009-4067). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved March 20, 2011, from http://ies.ed.gov/ncee/wwc/PracticeGuide.aspx?sid=12 Lipton, L., & Wellman, B. (2012). Got data? Now what? Bloomington, IN: Solution Tree Press. Love, N. (2009). Using data to improve learning for all: A collaborative inquiry approach. Thousand Oaks, CA: Corwin Press. Mandinach, E. B., & Gummer, E. S. (2012). Navigating the landscape of data literacy: It is complex. San Francisco, CA: WestEd. ThinkReliability. (2011). Root cause analysis. Retrieved February 12, 2014, from: http://www.thinkreliability.com/Root-Cause-Analysis-CM-Basics.aspx References