Using School Data for Problem-Solving

1. Using School Data for Problem-Solving Don Kincaid, Ed.D. Brian Gaunt, Ph.D. Shelby Robertson, Ph.D.

2. Questions • Why are data essential for problem-solving at the school level? • How do we identify the critical data needed for solving academic and behavior concerns? • How can the 4 step problem-solving model guide the use of academic and behavioral data in a systematic and common way across a district? (i.e., guiding questions).

3. Why are Data Essential for Problem-solving?

4. …for planning? “Research has found that up-front planning helped make data collection and use more efficient in many case study schools by clarifying what data were needed, aiding with integration of multiple data sources, and ensuring that data collection processes were on track (Keeney, 1998; Lachat, 2001).” -Kerr, et al., (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons learned from three urban districts. American Journal of Education, 112, 496-520.

5. …for legal reasons? • Some data we are required to collect (e.g., NCLB, 2002): • But why? • What questions does this “required data” potentially help answer? • Can this data be used in our problem-solving efforts? • Schools are required to use “evidence”, collect and analyze data, and use those data to make decisions for education improvement. (Coburn & Talbert, 2006; Halverson et al., 2005; Honig & Coburn, 2008; Mills, 2011; Young, 2006)

6. …to identify solutions to problems!!! • Not all data we are asked to collect will be useful for solving local problems. • Compliance vs. problem-solving • Main question: What data should we collect for problem-solving. • Tied to what questions are being asked • Types of data: Screeners, diagnostic measures, progress monitoring measures, outcome or summative measures. • Each has a function and potential to answer particular types of questions.

7. Decision-making – Need Data • Assumptions: • The appropriate data are needed for school level problem-solving • No matter how useful the data may be, they are NOT useful if they are not used. • Data Chaos! • To teach others about data we need to separate • Management (infrastructure-data system) • Use (analysis and decision-making)

8. How do we know what critical data are needed?

9. Consensus on Data?Coburn & Talbert, 2002 • What does “data-based problem-solving” mean to you? • Do you know what to collect? • Do you know why it is being collected? • Do you know what the data are used for? • Do you know what to do after you have the data? After you analyze it? • How aligned is your assessment system with answering key questions and how valuable are your district’s data to solve what problems?

10. Are your data useful? For what? Data source(s) should: • provide sufficient information to select appropriate services and supports. • allow you to group students with similar needs • match the nature of the problem, the target responses/knowledge identified for change, and key problem-solving questions.

11. Data Management: Let’s Redefine “Data System” • Prepare to Collect • Collect the data • Organize the data • Summarize the data • Share the data • Support the team to use the data • Analyze the data • …Make a decision!

12. What questions should we be asking to solve problems?

13. Problem Solving Process Define the Problem What Do We Want Students to KNOW and Be Able to DO? Evaluate Did It WORK? (Response to Intervention –RtI) Problem Analysis Why Can’t They DO It? Implement Plan What Are WE Going To DO About It?

14. Florida’s Guiding Questions Step 1 – Problem ID • What do we expect out students to know, understand, and do as a result of instruction? • Do our students meet or exceed these expected levels? (How sufficient is the core?) • Are there groups for whom core is not sufficient? Step 2 – Problem Analysis • If the core is NOT sufficient for either a “domain” or group of students, what barriers have or could preclude students from reaching expected levels? Step 3 – Plan Development and Implementation • What strategies or interventions will be used? • What resources are needed to support implementation of the plan? • How will sufficiency and effectiveness of core be monitored overtime? • How will fidelity be monitored over time? • How will “good”, “questionable,” and “poor” responses to intervention be defined? Step 4 – Plan Evaluation of Effectiveness • Have planned improvements to core been effective?

15. Step 1: Problem Identification –Tier 1 • What do we expect our students to know, understand, and do as a result of instruction? • Do our students meet or exceed these expected levels? (How sufficient is the core?) • Are there groups for whom core is not sufficient?

16. Expectations for Behavior • 80% have 1 or fewer ODRs • Are the # of ODRs, ISS and OSS per 100 students higher than the national or district average? • Are the # of ODRs, ISS and OSS per 100 students decreasing? • Is attendance steady?

17. Expectations for Literacy & Math • Sunshine State Standards (SSS) • Grade-level expectations (GLE) • Objectives and Goals of GLEs • The standards are the curriculum. • Tier 1 data: AYP (state test-NCLB); State reading test (FCRR/FAIR) • State assessments based on SSS. • Additional, district specific?

18. Annual Yearly Progress (AYP)

19. Step 1: Problem Identification –Tier 1 • What do we expect our students to know, understand, and do as a result of instruction? • Do our students meet or exceed these expected levels? (How sufficient is the core?) • Are there groups for whom core is not sufficient?

20. Do 80% of students exhibit appropriate behavior?

21. Do 80% of students exhibit appropriate behavior?

22. During the current year, does the school have students with 2 or more ODRs by October 1?

23. Are the # of ODRs, ISS and OSS per 100 students higher than the national or district average? • National Average for MS is .05 per 100 students

24. Are the # of ODRs, ISS and OSS per 100 students decreasing?

25. Are the # of ODRs, ISS and OSS per 100 students decreasing?

26. Is attendance steady?

27. Sources of Data

28. Utilizing Common Assessment Data to Understand Student Needs

29. Class Recommended Level of Instruction Report This report provides a summary of the students’ overall progress. It can be used to get an overall sense of instructional levels in the class and to calculate the Effectiveness of Core Instruction (ECI) index and the three Effectiveness of Intervention (EI) indices.

30. How sufficient is the core?

31. How sufficient is the core?

32. Academic Sufficiency

33. Step 1: Problem Identification –Tier 1 • What do we expect our students to know, understand, and do as a result of instruction? • Do our students meet or exceed these expected levels? (How sufficient is the core?) • Are there groups for whom core is not sufficient?

34. Are there groups of students for whom the Tier 1 Core is not sufficient?

35. Are there groups of students for whom the Tier 1 Core is not sufficient?

36. Are there groups of students for whom the Tier 1 Core is not sufficient?

37. Are there groups for whom core is not sufficient?

38. Are there groups for whom core is not sufficient?

40. Step 2 – Problem AnalysisTier 1 • If the core is NOT sufficient for either a “domain” or group of students, what barriers have or could preclude students from reaching expected levels?  • Why are some students not successful (Initial Hypotheses)?

41. What potential barriers have precluded us from achieving expected performance levels? Lack of… • Common Assessments • Common Planning • Ongoing Progress Monitoring • Curriculum Mapping Aligned with NGSSS and Common Assessments • Resource Availability • Administrative Support • Professional Development

42. Alignment with Standards and Across Grade/School Levels, Relevancy to Students’ Personal Goals, Content, Pacing, Progression of Learning, Differentiation Reinforcement Preferences, Perceptions of Competence and Control, Perceived Relevancy of Instruction/Education, Integration and Affiliation with School, Academic/Social-Emotional Skill Development Cognitive Complexity of Questions and Tasks, Gradual Release of Responsibility, Appropriate Scaffolding, Connection to Students’ Personal Goals, Interests and Life Experiences Reward/Consequence System, Visual Cues, Climate/Culture, Quality of Student/Adult Relationships, Quality of Peer Relationships, High Expectations for ALL Students, Collaboration and Voice

43. Hypothesis= Instructional

44. Step 2: Problem Analysis –Tier 1 1. Instruction • Are best practices in instruction being delivered to those students? • Is instruction being delivered in sufficient amounts or as often as necessary? 2. Curriculum • Are lesson plans in alignment with the appropriate core standards/expectations? • Are the curricular materials being used with fidelity or as designed? • Does staff have the knowledge and skills to utilize the curricular materials in alignment with grade-level/school-wide standards or expectations? 3. Environment • Do all staff and students know the school-wide behavioral expectations? • Are they being used consistently across all settings? (e.g., school climate)? • Are the school-wide behavioral expectations in alignment with the school/district missions? • Are best practices in classroom management being utilized and in alignment with the school-wide behavioral expectations? 4. Learner • Are students accessing the available instruction? (e.g., attendance) • Are students “actively engaged” in classroom instruction? • Do students perceive having a positive relationship with their school/teachers?

45. Step 3: Plan Devel. and Implementation–Tier 1 • What strategies or interventions will be used? • What resources are needed to support implementation of the plan? • How will sufficiency and effectiveness of core be monitored overtime? • How will fidelity be monitored over time? • How will “good”, “questionable,” and “poor” responses to intervention be defined?

46. Key Considerations • Utilize existing tools and resources whenever possible. • Align strategies and interventions specifically to identified barriers which preclude student success within core instruction. • Select research-based strategies and interventions to address identified barriers. • Communicate the “compelling why” of interventions with teachers, parents, and students.

47. Mentoring programs Goal Setting & career planning support Frequent progress reports Targeted rewards Mandatory study hall Mandatory homework help Study skills classes Targeted, differentiated instruction Additional instructional time Pre-teach essential skills, content, and vocabulary Review/Reteach prerequisite skills to address the learning gap Prevention (requires vertical articulation with middle/elementary school and early identification of at-risk students) Intervention Linked to Underlying Barrier Disengaged Learners Failed Learners CAUTION: Failed Learners often become disengaged over time and may require both categories of intervention support

48. Tier 1/Universal PBS: Specific RtI:B Action Plan Critical Elements: PBS Team; Faculty Commitment; Discipline Procedures; Data Entry & Analysis; Expectations & Rules; Reward/Recognition Program; Lesson Plans; Implementation Plan; Classroom Systems; Evaluation

49. Sources of Data

50. Selecting Research-Based Strategies and Interventions for Academics and Behavior