An Overview of a Problem Solving Model for Decision Making

1. An Overview of a Problem Solving Model for Decision Making A module for pre-service and in-service professional development MN RTI Center Author: Kerry Bollman, SSP, NCSP www.scred.k12.mn.us click on RTI Center

2. MN RTI Center Training Modules This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center for use in preservice and inservice training: 2

3. Overview • Historical Context and Evolution of Problem Solving Models • Problem Solving within a Response to Intervention Framework • Problem Solving Steps and Questions 3

4. Problem-Solving Steps and Questions 2. Problem Analysis Why is the problem occurring? 1. Problem Identification What is the discrepancy between what is expected and what is occurring? 3. Plan Development What is the goal? What is the intervention plan to address this goal? How will progress be monitored? 5. Plan Evaluation Is the intervention plan effective? 4. Plan Implementation How will implementation integrity be ensured?

5. Historical Application of a Problem Solving Approach • Cascade of Services (E. Deno, 1970) • Behavioral Consultation (Bergen & Kratochwill, 1990) • In the “early days,” the problem solving model was predicated on the n=1 approach. • Problem Solving approach promoted many improvements to service delivery relative to a traditional SPED testing model.

6. Problem Solving System Addressed Many School Problems

7. Problem Solving System Addressed Many School Problems

8. However, Not Yet a Perfect System • Five Major Challenges (Tilly, 2008) • Efficiency • More reactive than proactive • Capacity of individual teachers to implement multiple interventions. • Perception that problem solving continues to be the way to “get students into” special education. • Reengineering the problem solving model to incorporate new developments in research and practice.

9. Continued Evolution of the Problem Solving Model • Moving away from an N=1 model toward one in which the problem solving model is to be used for all students in the system, not just those who struggle. • Examination of core curriculum • School-wide data collection for purposes of screening and program evaluation • Ability to implement data-based groupandindividual interventions (continued…)

10. Continued Evolution of the Problem Solving Model (cont.) • Application of the problem solving model equally to both academic and social behavior concern areas • Move away from sole use as an expert driven consultation model, toward a model that supports collaboration between general and special education staff 10

11. Problem Solving Within the Context of a School Wide Response to Intervention Framework Assessment Instruction Response to Intervention Problem-Solving & Organization

12. Processing Activity • List three important improvements that a problem solving model offers to schools as compared to a traditional service delivery model • List three reasons why utilizing problem solving as a systems improvement model rather than solely a “1 student at a time” model benefits schools • Discuss how the identification of students who may need intervention support through a review of systems wide screening data as opposed to relying entirely on individual teacher referral may reduce bias.

13. Decision-Making Model Used by Problem-Solving Teams

14. Problem-Solving Steps and Questions 2. Problem Analysis Why is the problem occurring? 1. Problem Identification What is the discrepancy between what is expected and what is occurring? 3. Plan Development What is the goal? What is the intervention plan to address this goal? How will progress be monitored? 5. Plan Evaluation Is the intervention plan effective? 4. Plan Implementation How will implementation integrity be ensured?

15. Step 1: Problem Identification • Question: What is the discrepancy between what is expected and what is occurring? 1. List problem behavior(s) and prioritize. 2. Collect baseline data on primary area of concern (target student and comparison). • Record Review • Interview • Observation • Testing 3. State discrepancy between target student performance and peer or expected performance.

16. Problem Identification Key Points • Collect & analyze regular school-wide screening data on most common referral concerns for efficient problem identification in these areas • A decision must be made about how to define “expected” (local norms / national norms / criterion) so teams know which students to identify as at-risk (continued…)

17. Problem Identification Key Points (cont.) • One concern must be prioritized at a time • Trying to take on everything at once often results in getting nothing done • Concern needs to be stated measurably • Avoid “problem admiration” during this phase • Data from a variety of sources should converge to certify the problem • Screening data + other available evidence 17

18. Problem Identification Key Points (Cont.) • Converging Evidence • Multiple sources of data that each point to a consistent conclusion • RIOT = Review, Interview, Observe, Test • Consider all these sources when seeking convergent evidence to certify a problem • ICEL = Instruction, Curriculum, Environment, Learner • Consider all these domains when seeking convergent evidence

19. Processing Activity Choose at least 1 pair above, and brainstorm about the information you could collect. Example: Reviewing Curriculum: What core and supplemental materials have been used? How many lessons are included on the topic of difficulty? Do the materials provide sufficient modeling and practice? Is mastery of this skill expected at this grade level? Was homework included?

20. Problem Identification in Practice • Step 1: Screen all students • Step 2: Identify all students at risk (all those who scored below the target score established for the assessment • Step 2a: For students not at risk, plan to re-screen in 3-4 months • Step 3: For all students identified as at risk, consider other data sources across domains to see if you have convergent evidence of a problem (RIOT & ICEL). • Step 3a: For students where no convergent evidence of a problem is found, confirm that strong core curriculum is in place and consider periodic monitoring • Step 4: For students where convergent evidence of a problem is found, proceed with problem solving to develop an intervention • Plan for with group interventions where reasonable

21. Step 2: Problem Analysis Problem Analysis is the process of gathering relevant information in the domains of the instruction, curriculum, environment, and the learner (ICEL) through the use of reviews, interviews, observations, and tests (RIOT) in order to evaluate the underlying causes of the problem. Heartland AEA II

22. Step 2: Problem Analysis • Question: Why is the problem occurring? • Consider what you know about the target behavior that is relevant to determining why the problem is occurring and a possible solution • Based on what you know list possible causes for the student’s problem (hypotheses) • Consider all domains (Instruction, Curriculum, Environment, Learner) • Differentiate between skill and performance problems • Determine situations in which the problem is most and least likely (continued…)

23. Step 2: Problem Analysis(cont.) • For each hypothesis, list supporting and non-supporting data • Narrow down to the most validated and alterable hypothesis • Collect any additional data you need to validate the hypothesis that the team considers to be the most likely • Need at least 2 pieces of convergent evidence, one should be quantitative 23

24. Problem Analysis Key Points There can be several possible underlying reasons why a student is doing poorly in an academic area. It is crucial to determine the reason(s) for poor performance in order to select an appropriate intervention: • They don’t want to do it - The student has the necessary skills but lacks the motivation to complete the task • They haven’t had enough practice - The student possesses the necessary skills but is not yet fluent and automatic in those skills

25. Problem Analysis Key Points • They haven’t had enough instruction The student lacks the necessary skills to perform the academic task & needs additional instruction, modeling, and feedback. • They haven’t had to do it in that way before - The student needs support to generalize skills to new settings • The skill is too hard - the student needs instruction in pre-requisite skills 25

26. Problem Analysis: Processing Activity • How would you find out whether the cause of an academic problem for a student was due to lack of practice with the skill, or due to a need for additional instruction?

27. Step 3: Plan Development • Question: What is the goal? • Write the goal, a measurable statement of expected outcomes. • Question: What is the intervention plan to address the goal? • Define logistics (e.g., what strategies/procedures will be used, when and how often the intervention will occur, who will implement the intervention and where it will be implemented, and when it will begin). • Question: How will progress be monitored? • Define logistics (e.g., what materials are used, when and how often data will be collected, where data will be collected, and who is responsible). • Decide on decision-making rules for plan evaluation.

28. Plan Development Key Points • For goal setting, it is helpful to determine rate of growth for goal that would result in a reduction of the discrepancy of student performance • Oral Reading Fluency: 2 words per week or to local spring target • Written Expression: 1/2 CWS per week or to local spring target • Math Facts: 1/2 fact per week or to local spring target • Behavior: 10% improvement per week (continued…)

29. Plan Development Key Points (cont.) • Any intervention idea chosen for the student should be scientifically research based • Within a district, developing a master list of interventions used for which staff have training and necessary materials is helpful (continued…) 29

30. Plan Development Key Points (cont.) • Access resources to assist with development of interventions list • MN RTI Center intervention list http://www.scre.k12.mn.us click on MN RTI Center • http://ies.ed.gov/ncee/wwc/ • http://www.fcrr.org • http://www.interventioncentral.com • Important for teams to understand the difference between an intervention and a modification

31. Plan Development Key Points (cont.) • Role of master schedule in planning interventions - planning time and staff • Tool chosen for progress monitoring must have adequate technical adequacy for this purpose - outcomes measurement • Teams must determine what represents adequate response to the intervention 31

32. Plan Development: Processing Activity • Generate a list of all the scientifically based interventions you are aware of • Could be scripted programs like DI Reading Mastery, or protocols like Incremental Rehearsal • Could be for academics as above, or like Check and Connect for social behavior concerns

33. Step 4: Plan Implementation • Question: How will implementation integrity be ensured? • Provide training and support to those implementing interventions. • Observe intervention in action. • Make adjustments to intervention plan if needed. • Collect and graph data on intervention goal.

34. Plan Implementation Key Points • Intervention Scripts & Training • Increases likelihood that specifics of intervention will be well understood by those performing the intervention • Are preferred by interventionists rather than global intervention descriptions • Training should include modeling, practice, and feedback with adults prior to use with students

35. Plan Implementation Key Points • Integrity checks: It is impossible to evaluate the success of a plan if the team is not certain that the plan was implemented as designed • Did the program happen as planned at least once? (formal observation) • Do you have data to indicate that the student participated fully in the intervention? (attendance, time logged in, accuracy of participation in intervention)

36. Example Integrity Script Script developed at St. Croix River Education District

37. Step 5: Plan Evaluation • Question: Is the intervention plan effective? 1. Use data to determine student progress. 2. Evaluate intervention acceptability. 3. Determine as a team what to do next.

38. Example: Instruction is working for student. Reading Sunnydale EastRiser Elementary Smith District 09-10 Jacob Year School Teacher Name By May 31, Jacob will read 110 words correct per minute on 3rd grade material. Goal Statement Expected Level of Performance #1 #2 #3 #4 Service Providers Parent Participation 120 Baseline 110 Goal 100 I-Phonics for Rdg. 90 80 70 60 Words correct per min. 50 40 30 20 10 0 M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M Day Heartland AEA II

39. Student Improvement is Job #1 Goal Area Math Shadyside Northern Heights Jackson District 09-10 Pam Year School Teacher Name By May 31, Pam will score 45 digits correct per 2 minutes on 5th grade mixed math probes. Goal Statement Expected Level of Performance #1 #2 #3 #4 Service Providers Parent Participation Goal 48 Baseline 44 I-PALS 40 36 32 28 Digits correct per 2 min. 24 20 16 12 8 4 0 M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M Day Example: Instruction is NOT working for the student. I-PALS + Reteach division Heartland AEA II

40. Plan Evaluation Key Points • Ensure agreement on implementation integrity prior to evaluating outcomes of an intervention • Evaluate student outcomes only for interventions that have been fully implemented • Plan Evaluation does not happen without a graph • Determination of next steps: teams might: • Identify a new problem • Consider a different hypothesis for the same problem • Plan a new intervention to address the same problem and hypothesis • Rework intervention to be able to achieve implementation integrity • Celebrate student success!

41. At Plan Evaluation, teams may stop, or re-enter the cycle at any point 1-4 2. Problem Analysis Why is the problem occurring? 1. Problem Identification What is the discrepancy between what is expected and what is occurring? 3. Plan Development What is the goal? What is the intervention plan to address this goal? How will progress be monitored? 5. Plan Evaluation Is the intervention plan effective? 4. Plan Implementation How will implementation integrity be ensured?

42. Case Study - Problem Solving Process Beginning School Wide Continuing with Small Group Intervention Resulting in Individual Intervention

43. Step 1:Problem Identification Question: What is the discrepancy between what is expected and what is occurring? • Third grade team met in September to review fall R-CBM and NWEA MAP Reading data • Identified a group of 14 students in general education with scores of concern • Below fall grade 3 targets on R-CBM and/or MAP • Teacher report (last year, this year) • Statements of discrepancy based on individual scores relative to expectation

44. Fall Grade 3 Data:Students of Concern

45. Step 2: Problem Analysis • Question: Why is the problem occurring? • Team noticed that most of these students demonstrate high error rates in oral reading • Review of records indicate pattern of performance across years • Teachers report poor phonics skill mastery among these students • Hypothesize that these students are discrepant because they need additional instruction in phonics

46. Step 3: Plan Development • Core instruction response K-2 • Considered allotted time for reading instruction • Worked on increasing time within reading classes for students actively interacting with text • Reviewed scope and sequence for phonics and added more modeling and practice on these skills to core program • Collect grade-wide screening data and set goals to increase percentages of students meeting spring targets by grade level

47. Step 3: Plan Development • Question: What is the goal? • Team sets a goal that in 10 weeks, all identified students will increase their fluency on 3rd grade passages by 15 WRC per minute. • 1.5 words per week growth from baseline • Question: What is the intervention plan to address the goal? • Replace silent reading time each day with small group focused instruction in phonics. • Chose Phonics for Reading level 2 • Set up extra incentive program for independent reading at home with parents for these 10 weeks • Question: How will progress be monitored? • Weekly monitoring of CBM.

48. Step 4: Plan Implementation • Sped teacher provided 2 classroom teachers with training on phonics intervention • Students moved to one of these two classes to receive intervention each day • Half of non-participating students in each classroom of teacher providing intervention moved to another classroom for silent reading • School counselor from team conducted 1 observation for implementation integrity for each teacher • Teacher kept track of time spent on intervention • Sped para collected weekly progress monitoring data

49. Step 5: Plan Evaluation 100 90 80 70 60 50 40 30 Matt Baseline 9/17 9/24 10/1 10/8 10/15 10/22 10/29 11/5 11/12

50. The Individual Student • Matt is in the 3rd grade group of students receiving additional phonics instruction in place of silent reading • Grade level team meet in November to review outcomes data from group intervention. • Teachers note that Matt is not making adequate progress toward his goal despite this intervention. • Make a referral to the building-based problem solving team