RTI Module 3 Data Analysis and Team Problem Solving Process

1. RTIModule 3Data Analysis and Team Problem Solving Process Marika Ginsburg- Block, Ph.D. University of Delaware marika@udel.edu Todd Gravois, Ph.D. University of Maryland tgravois@verizon.net And The Delaware Department of Education

2. Agenda: 8:00-8:30 Registration/ Networking 8:30-9:00 Welcome/ Homework Review 9:00- 10:15 Problem Solving as a Core Skill RTI 10:30- 11:30 Demonstration of Effective Problem Solving at Tier 1 11:30-12:00 Role Alike Activity 12:00-12:45 Working LUNCH- with District Teams 12:45- 1:30 Data Analysis & Decision Making within Problem Solving 1:30- 2:30 Case Study with Application of Problem Solving & Analysis 2:30- 3:15 District Team Activity 3:15- 3:30 Homework Assignment 3:30- 3:50 RTI Framework Debriefing 3:50- 4:00 Evaluation

3. Purpose of Training Modules • RTI is not a program; RTI is a process • State provides a framework and process • Through regulations • Through professional development • Through technical assistance • District/School develops unique implementation • RTI implementation must take into consideration unique characteristics of local culture • RTI should build on existing systems/initiatives • District Leadership • Diversity in contribution/input/skills • Distribution in workload • Everyone is knowledgeable and supportive • Share framework and provide guidance for school implementation

4. RTI Framework Where have we been? • Established District Leadership Team • Overview of RTI • Needs Assessment of District and School Level Implementation • Tier 1 Framework • Assessment Framework • Assessments (December) • Data Management (December) Where are we going? • Team Problem Solving (February) • Data Analysis (February) • Interventions (April) • SLD Determination (May) • Secondary Focus (2008-2009)

5. Three -Tier Model of School Supports Intensive Quality Instruction Quality/ Individualized/ Documented/ Interventions Quality Classroom Instruction Decisions for those students not benefiting from interventions Decisions for those students not benefiting from the core curriculum 5

6. Let’s Be Clear….. Individualized problem solving and decision making is critical … but not synonymous with how programming is delivered. Individualized does not equate to individual instruction or individual interventions.

7. Three major considerations for problem solving and data analysis: • Where in the RTI process should problem solving occur? • What constitutes effective problem solving? • How does data analysis relate to effective problem solving

8. Where should problem solving and data analysis focus? Decisions for those students not benefiting from interventions Intensive Quality Instruction Decisions for those students not benefiting from the core curriculum Quality/ Individualized/ Documented/ Interventions Quality Classroom Instruction There will be times when problem solving and data analysis will be helpful in making decisions to identify students needing more assistance. 8

9. However, there will be times when problem solving and data analysis will be critical in determining the quality of instruction being delivered. Decisions for those students not benefiting from interventions Intensive Quality Instruction Decisions for those students not benefiting from the core curriculum Quality/ Individualized/ Documented/ Interventions Quality Classroom Instruction 9

10. “Fundamentally, RTI assumes that producing changes in student responding over time is important, which establishes the instructional task for educators. School-based intervention teams must be able to ensure that instruction is adapted correctly to students’ level of skill proficiency.” Daly, et al., 2007

11. There will be times when problem solving and data analysis will be critical in determining the quality of instruction being delivered. Intensive Quality Instruction Quality/ Individualized/ Documented/ Interventions Quality Classroom Instruction 11

12. Intensive Quality Instruction Quality/ Individualized/ Documented/ Interventions Quality Classroom Instruction 12

13. So, what is “quality” instruction? Intensive Quality Instruction Quality/ Individualized/ Documented/ Interventions QualityClassroom Instruction How do we define quality instruction? 13

14. Multiple Influences on Learning:(Gravois, Gickling & Rosenfield, 1999) (50-60%) Prior Knowledge Student Match=Success Instruction Task (25-35%) (5-15%)

15. Influences on Learning:High Achievers (80-90%) Prior Knowledge Student Match=Success Instruction Task (5-10%) (5-10%) 15

16. Influences on Learning:Low Achievers (10-20%) Prior Knowledge Student Match=Success Instruction Task (40-45%) (40-45%) 16

17. Importance of Task Long raidans were forming when Matthew arrived. He tried to phindate the amount of time it would take to get to the cornvorster. Vort it would be too long, plast he would miss the game. He varaxated for a moment until the raidans became even longer. He decided that he would ordrul in the raidan opet see vort it would start moving more expedititiously. No sooner had he started fleedjuul, when it began mostulag quite hard. Matthew became disgusted, zipped up his ornaforger, then walked back to his car. He drove home in the mostul. By the time he put the car in the garage, the mostul was droim poet the faetos was out. Matthew was doubly disgusted now. Suddenly, he went inside to watch the game. He turned on the television set but nothing happened. Matthew said to himself, “what a lousy frol”.

18. Three major considerations for problem solving and data analysis: • Where in the RTI process should problem solving occur? • What constitutes effective problem solving? • How does data analysis relate to effective problem solving

19. Activity 1: What is effective problem solving? • Think-Write-Share • Working individually first think about how you approach a problem. • Quickly write the steps of problem solving you typically follow-- try to be specific • Share with a partner from your school. Compare and contrast. What are the implications of varied approaches to problem solving?

20. Researched Characteristics of Effective Problem Solving • Explicit and Observable Descriptions • Collection of Baseline Data • Specification of Measurable Goals • Specification of Intervention Components • Integrity of Intervention Implementation • Graphing of Data to Monitor Progress • Evaluation of Intervention by Comparing Graphed Data to Baseline/ Goals Donovon & Cross, 2002

21. Researched Characteristics of Effective Problem Solving (cont.) • Problems are best defined as a discrepancy between a desired state and what is occurring. • When students are experiencing problems, one key to finding effective instructional approaches is to conduct analysis of why we believe students are not proficient. This analysis must focus on instructionally relevant and changeable variables. • A plan is developed that is • Goal directed • Based on an analysis of the problem • Identifies specific What? When? and How? • Specifies method for monitoring progress • Documents the plan • Fits the resources and values of people in the setting • Progress is monitored frequently and repeatedly, changes are timely, plan is adjusted based upon data. NASDSE 2005

22. Researched Characteristics of Effective Problem Solving (cont.) • Increased specificity and integrity of the process linked to greater goal attainment (Levinsohn & Rosenfield, 2000) • Increased accuracy and integrity of documenting the problem solving process the more likely to achieve the goals established (Fudell, 1992; LaFleur & Rosenfield, 2005)

23. Researched Characteristics of Effective Problem Solving (cont.) “It is important to ensure that all factors (e.g., curriculum, effective instruction, school and classroom environment) have been examined prior to assuming that a student factor (or disability) are responsible for student performance.” NASDSE 2005

24. Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation Strategy/ Intervention Evaluation Follow-up and Re-design

25. Problem Solving Process Problem Identification & Problem Analysis Specific and observable concerns Assessment of instructional conditions Data Analysis Principles Prioritized/ Targeted Concerns Baseline Data (direct assessment of concern) Specific and measurable goal Charting and graphing of data

26. Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Academic: • Conducted under instructionally matched conditions • Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) • Plan for progress monitoring Behavior: • Conducted under instructionally matched conditions • Application of researched behavior principles • Contingency management • Plan for progress monitoring

27. Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation • Implementation integrity must be considered • Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks.

28. Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation Strategy/ Intervention Evaluation • Charting and graphing of data (at least weekly) • Continued comparison of data with baseline and goals

29. Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation Strategy/ Intervention Evaluation Follow-up and Re-design - Recognition that refinement and “tweaking” are necessary parts of effective problem solving

30. Activity 2: Problem Solving Process • Using the criteria for effective problem solving, listen to the case presentation. • Note those components of effective problem solving that were addressed and which were not addressed. • For any aspects not addressed, note how might those tasks be accomplished. Examining ProblemSolving

38. Afternoon Agenda What’s still to come? • Cases taken across the three-tiered model • District team planning time • How to evaluate problem solving in your district?

39. Activity 3: Role Alike Reflection • Specific to problem solving and data analysis, what skills/ expertise does your training/ position bring? • What skills/ expertise do you think will be required for effective problem solving and data analysis? • What role do we play currently? How do we envision our role in how RTI will be implemented in our district? • How will we be involved in maintaining integrity/ fidelity to quality instruction?

40. Lunch time! During your working lunch, please: • Follow up with your district team • Share what you learned from the Role Alike Activity

41. Three major considerations for problem solving and data analysis: • Where in the RTI process should problem solving occur? • What constitutes effective problem solving? • How does data analysis relate to effective problem solving?

42. How does Data Analysis Relate to Effective Problem Solving Problem Identification & Problem Analysis • Specific and observable concerns • Assessment of instructional conditions • Data Analysis Principles • Prioritized/ Targeted Concerns • Baseline Data (direct assessment of concern) • Specific and measurable goal • Charting and graphing of data Strategy/ Intervention Design Academic: • Conducted under instructionally matched conditions - Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) - Plan for progress monitoring • Behavior: • Conducted under instructionally matched conditions - Application of researched behavior principles - Contingency management - Plan for progress monitoring Strategy/ Intervention Implementation • Implementation integrity must be considered • Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks. Strategy/ Intervention Evaluation • Charting and graphing of data (at least weekly) • Continued comparison of data with baseline and goals Follow-up and Re-design - Recognition that refinement and “tweaking” are necessary parts of effective problem solving

43. How does Data Analysis Relate to Effective Problem Solving Problem Identification & Problem Analysis Specific and observable concerns Assessment of instructional conditions Data Analysis Principles Prioritized/ Targeted Concerns BaselineData(direct assessment of concern) Specific and measurable goal Charting and graphing of data Strategy/ Intervention Design Academic: Conducted under instructionally matchedconditions - Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) - Plan for progress monitoring Behavior: Conducted under instructionally matched conditions - Application of researched behavior principles - Contingency management - Plan for progress monitoring Strategy/ Intervention Implementation Implementation integrity must be considered Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks. Strategy/ Intervention Evaluation Charting and graphing of data (at least weekly) Continued comparison of data with baseline and goals Follow-up and Re-design - Recognition that refinement and “tweaking” are necessary parts of effective problem solving

44. Guiding PrinciplesData Analysis: • Occurs at all Tiers and Stages of Problem Solving • Must match the purpose • Involves use of multiple sources of data • Is only good as the measure being used • Requires data tools and involves multiple techniques • Involves thoughtful application of decision rules • Depends upon the skills and expertise of the professional making decisions • Results in hypotheses, not certainties • (Sources: Heartland AEA (2007))

45. Activity 4: Case of Ms. Brown • Applying Effective Problem Solving and Analysis: Case Study (Tiers 1 - 3)

46. Case of Ms. Brown: Background: Part 1 • RTI Regulations • Tier 1 screenings conducted 3x annually; 1st screening within 2 weeks of school • Children at or below 25th%ile will receive Tier 2 services • Children above 25th%ile but below benchmark shall receive differentiated, needs-based instruction; progress monitored every 2 weeks until on trajectory; after 6 weeks Tier 2 or continued monitoring • Assessment Tools used in this case • Oral Reading Fluency • Nonsense Word Fluency • Data analytic terms • 25th %ile & benchmarks • Level of performance & rate of improvement • Dual discrepancy

47. Part 1- Small Group Discussion Questions: • Data on page 4, Questions on page 5 (only work on Q. 1 &2) • 1. What hypotheses do you have about the basis of the student difficulties illustrated here? • 2. What data would you need to test each of your hypotheses (complete the chart that follows)?

48. Case of Ms. Brown: Background: Part 2 • RTI Regulations • Tier 2 delivered in small group setting, 3 times per week for 30 minutes for 6 weeks, weekly monitoring against benchmarks • After 6 weeks IST reviews progress, including fidelity of implementation, pacing, appropriateness of instructional groupings • Team may request additional assessments, modify Intervention and continue Tier 2 for 6 more weeks, determine Tier 3 intervention is required

49. Part 2- Small Group Discussion Questions: • Data on pages 7-8, Questions on page 9 (only work on Q. 1 &2) • 1. Based on the progress monitoring data provided, describe the effectiveness of Tier 2 instruction for each of these three students. Consider student level of performance on CBM measures (as compared to the benchmarks) and rate of improvement towards the benchmarks (i.e. their actual trendline compared to their aimline). • 2. Based on the intervention fidelity calendar, describe the implementation fidelity of this Tier 2 instruction. Is there additional data that you’d like to see?

50. Case of Ms. Brown: Background: Part 3 • RTI Regulations • After total of 12 weeks of Tier 2 intervention and no progress or not on trajectory toward benchmarks child shall begin receiving Tier 3 intervention • Tier 3 delivered in smaller group settings, at least 5x per week for 30 min, for 6 wks, weekly monitoring • After 6 wks of Tier 3 and no progress IST shall refer for initial evaluation