Collaborative Intervention Design for Education Improvement

1. Intervention Design A collaborative project between the Florida Department of Education and the University of South Florida FloridaRtI.usf.edu

2. Advance Organizer Update: Consensus, Infrastructure, Implementation Data Review: Problem ID, Problem Analysis Linking Problem Analysis to Intervention Intervention Design Intervention Content Intervention Plan Intervention Integrity, Support, Documentation Integrating Tiers of Intervention

4. Intervention Selection/Design & Implementation in Context Evaluate Intervention Effectiveness Monitor Progress Analyze the Problem Identify the Problem Implement Intervention Select/Design Intervention J L Timeline

5. Review Consensus Infrastructure Implementation

6. Discussion - Consensus • What activities are occurring to facilitate consensus building among school and district staff? • What activities need to occur to better facilitate consensus building among school and district staff? • What consensus building activities should be given top priority between now and the end of the school year?

7. Discussion - Infrastructure • What activities are occurring to facilitate the development of the infrastructure needed to implement a PS/RtI model? • What activities need to occur to better facilitate the development of the infrastructure needed to implement a PS/RtI model? • What infrastructure development activities should be given top priority between now and the end of the school year?

8. Discussion - Implementation • Given answers to the questions addressing consensus & infrastructure, what are implications for facilitating implementation of PS/RtI model in your building? • What components of model can be targeted for implementation between now and end of school year given the levels of consensus and infrastructure? • What components of model would need to be targeted for implementation later to allow more time to build support from staff and capacity (e.g., assessments, technology, training, policies, procedures) to implement?

9. Review Previous Skill Assessments • Problem ID • Gap between desired performance and current performance • Problem Analysis • Most likely barriers to desired performance • Using what you now understand of Problem ID and Problem Analysis, consider your Tier One data, identify a problem and analyze it. Later, you’ll link your Problem Analysis to the development of a comprehensive intervention plan.

10. Hypothesis Prediction Problem Analysis Intervention Design The problem is occurring because ______________. If ___________ would occur, then the problem would be reduced.

11. Intervention Development • Interventions derived from effective problem-solving strategies • Research/knowledge used to develop hypotheses. • Verified hypotheses lead naturally to interventions • Interventions will be evidence-based if hypotheses are evidence-based

12. Types of Interventions • Skill Deficit • Student lacks skills to successfully complete task • Performance Deficit • Factors interfering with student’s capability of performing the skill

13. Intervention Development:Verified Hypotheses • H: Only 55% of second grade students are achieving reading fluency benchmark because the curriculum lacks fluency focus. • P: If we modify the curriculum to focus on reading fluency, then more students will achieve benchmark. • Intervention: Modification of second grade curriculum to include more reading fluency focus

14. Intervention Development:Verified Hypotheses • H: Brandon is unable to stay in his seat because he lacks the self-monitoring skills necessary for self-control • P: If we teach Brandon self-monitoring skills, then he will display improved self-control. • Intervention: Self-instructional and self-monitoring training

15. Intervention Development:Verified Hypotheses • H: Ernesto is unable to complete arithmetic seatwork on time because he has not acquired math facts at the automatic level • P: If we provide Ernesto with strategies to increase fact fluency, then he will complete work on time. • Intervention: • Provide Ernesto with strategies to increase fact fluency • Provide Ernesto more time until strategies to increase fact fluency have had time to work

16. Intervention Development:Verified Hypotheses • H: Gail is not attending school daily because she believes that all of the students she goes to class with exclude her from social activities • P: If we help Gail to restructure her beliefs about being excluded, then attendance will improve. • Intervention: • Cognitive restructuring regarding belief system • Identify students who would be willing to include Gail in their activities • Ensure adult prompting and monitoring of social activities

17. Intervention Development:Verified Hypotheses • H: Susanna is unable to comprehend at 5th grade level because she is disfluent in 5th grade reading material. • P: If we improve her fluency, then comprehension will improve. • Interventions: • Limit comprehension expectations to current fluency level • Develop hypotheses to explain low rate of fluency

18. Tiers of Intervention I & II

19. Tier I Intervention • TierOne- Examining “Universal” Interventions • Questions: • What percent of students are achieving district benchmarks? Effectiveness of instruction • How are these students doing compared to grade level benchmarks? GAP analysis • Hypotheses • Ho: These students have not had access to an effective learning environment. • Ho: These students have not been engaged in an effective learning environment.

20. Tier 1 Intervention • Consider altering whole group instruction, curriculum materials, instructional routine, independent practice (e.g., literacy/math centers) • Breadth of skill focus might vary • Group students based on skill data (data come from many sources) • Differentiate instruction based on grouping • Organize students based on skill performance • Higher performing, more students, • Lower performing, fewer students • Same amount of time, different use of that time

21. Tier 2 Intervention • TierTwo- Examining “Supplemental” Interventions • Hypotheses: • Ho: Student requires additional time for direct instruction • Ho: Focus of the curriculum must narrow • Assessment: • DIBELS, CBM, district assessments • Interventions: • Increase AET (90-120-180) e.g., K-3 Academic Support Plan • Narrow focus to fewer, barrier skills • District Supplemental Curriculum

22. Characteristics of Tier 2 Interventions • Available in general education settings • Opportunity to increase exposure (academic engaged time) to curriculum • Opportunity to narrow focus of the curriculum • Sufficient time for interventions to have an effect (10-30 weeks) • Often are “standardized” supplemental curriculum protocols – identified through a problem solving process

23. Tier 2 Intervention • First resource is TIME (AET) • HOW much more time is needed? • Second resource is CURRICULUM • WHAT does the student need? • Third resource is PERSONNEL • WHO & WHERE will it be provided?

24. Tier 2: Getting TIME • “Free” time--does not require additional personnel • Staggering instruction • Differentiating instruction • Cross grade instruction • Skill-based instruction • Standard Protocol Grouping • Reduced range of “standard” curriculum • After-School • Home-Based

25. Tier 2: Curriculum • Standard protocol approach • Focus on essential skills • Most likely, more EXPOSURE and more FOCUS of core instruction • Linked directly to core instruction materials and benchmarks • Criterion for effectiveness is ≈70% of students receiving Tier 2 will reach benchmarks

26. Tier 2: Personnel • EVERYONE in the building is a potential resource • Re-conceptualize who does what • Personnel deployed AFTER needs are identified • WHERE matters less and less • REMEMBER, student performance matters more than labels, locations and staff needs. • A school cannot deliver intensive (Tier 3) services to more than ≈7% of the population

27. Outline – Implementing An RtI System • Tier 2 Decision Making – • Identify less than proficient students • Administer additional brief assessments to examine performance profiles • Group students with like performance profiles for supplemental instruction • Provide supplemental instruction based on skill needs

28. Outline – Implementing An RtI System (continued) • Tier 2 Decision Making – • Monitor progress • Review student progress monitoring data at scheduled intervals • How successful are students in response to Tier 2 Interventions? • ≈70 - 80% is a good criterion • Modify supplemental instruction as necessary • Move students across tiers as data warrant

29. Job-Alike • What is your role with respect to the PS/RtI model at your school? • What is working well with PS/RtI? • What is not working well? • What is the best part when it comes to implementing PS/RtI at your school?

30. Intervention Content

31. Why Intervene? Problem Definition & Problem Analysis have revealed information the problem solving team will use to determine what and how the students need to be taught. The purpose of Intervention is to create an instructional match

32. Instructional Match = Explicit Evidence-Based Instruction in Targeted Skills within a Supported Learning Environment

33. Criteria for Interventions • Evidence-based • Delivered with Integrity • Scaled for Intensity • Implemented for Sufficient Time • Evaluated Frequently • Integrated Across Tiers

34. Evidence-Based SCIENTIFICALLY BASED RESEARCH - • means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs • Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001

35. Evidence-Based • Employs systematic, empirical methods that draw on observation or experiment • Involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions • Relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations. • Is evaluated using experimental or quasi-experimental designs… with appropriate controls… with a preference for random-assignment experiments • Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001

36. Evidence-Based • Ensures that experimental studies are presented in sufficient detail and clarity to allow for replication or, at a minimum, offer the opportunity to build systematically on their findings • Has been accepted by a peer reviewed journal or approved by a panel of independent experts through a rigorous scientific review • Section 9101(37) of ESEA, as amended by the No Child Left Behind Act (NCLB) of 2001 Practitioner journals or education magazines are not the same as peer-reviewed academic journals

37. More Less Measurement Frequency Evidence Base Measurement Precision Group Size Measurement Focus Depth of Problem Analysis Instructional Time Less More

38. Principles of Intervention Design Intervention is… • Planful- procedures to be applied are specified clearly and completely • Environmentally Focused- actions taken modify the environment not the individual • Goal Directed- the team writes an ambitious, yet attainable goal statement prior to intervention design

39. Principles of Intervention Design Intervention should be designed to: • Adjust what is being taught and / or • how it is taught

40. Intervention Plan

41. Components of an Effective Intervention Plan Address: • Persons Responsible • Skills Targeted • Implementation Arrangements • Measurement Strategy • Decision Making Rule In essence - who, what, when, where

42. Designing anIntervention Plan

43. Persons Responsible Specifically named individuals that will be responsible for: implementing the intervention, supporting the intervention, assessing the integrity of the intervention, and monitoring the effectiveness of the intervention

44. Target Skills & Instructional Strategies Explicitly define the skills to be taught and strategies that will be utilized. Select evidence based instructional strategies that are based on the defined problem and verified hypotheses in order to maximize likelihood of success.

45. Implementation Arrangements • Where • When • Frequency • Length of time • Materials

46. Intervention Plan • Must include: • Who is responsible? • What will be done? • When will it occur? • Where will it occur?

47. Design an Intervention Plan:Measurement Strategy Record information about: • Who is responsible for the on-going collection of data • The method of data collection-(ex.:probes/graphs, frequency counts/graphs) • The measurement conditions- (ex.: environmental factors are consistent) • The monitoring schedule- (ex.: establishing consistent measurement intervals)

48. Design an Intervention Plan:Decision-making Plan How do we decide if a plan is or isn’t working? Decisions will be made based on the following: • Level of skill • Rate of progress • Decision rule

49. Decision Rules: What is a “Good” Response to Intervention? • Positive Response • Gap is closing • Can extrapolate point at which target student will “come in range” of peers--even if this is long range • Questionable Response • Rate at which gap is widening slows considerably, but gap is still widening • Gap stops widening but closure does not occur • Poor Response • Gap continues to widen with no change in rate.

50. Positive Response to Intervention Expected Trajectory Performance Observed Trajectory Time