PSM/RtI Formal Evaluation Level 3 at Elementary School

1. PSM/RtI Formal Evaluation Level 3 at Elementary School Curriculum Based Assessment and/or Behavior Data

2. Objectives • Define and discuss the following tools used in Responsiveness to Instruction: • Curriculum-based Assessment and Norms • Baseline data, Goals and Aimlines, Progress Monitoring • Graphing data • Decision-making rules • Revisions of Hypotheses/Interventions (Level 2 e)

3. Curriculum Based Assessment (CBA) Data collection tools derived directly from the curriculum that student is expected to learn

4. Curriculum Based Assessment (CBA) • CBM is believed to reduce the gap between assessment and instruction • Aides teachers in improving instruction • Improved communication • Higher level of sensitivity • Administration time is shorter • More cost effective

5. To Use CBAs in RtI: Normative Data is Required • Why do we do it? How is it useful? • To provide comparison data on skill levels of students with the same curriculum and demographics of our county • When or how often do we do it? • Every 5 years • How do we do it? • Stratified Random Sample • Fall, Winter and Spring Windows

6. ACADEMICS Reading, Math and Written Language Probes; e.g., CBAs

7. Probes: Measuring Academic Skills • Reading: The Big Five Areas a Hierarchy • Phonemic Awareness • Phonics • Fluency • Vocabulary • Comprehension

8. Probes: Measuring Academic Skills • Math • Number Sense • Calculation • Reasoning • Written Expression • Conventions • Spelling

9. EVALUATION THATINFORMS INSTRUCTION • Baseline Data: How do the student’s skills compare to those of peers (normative sample) • Goals and Aimlines: Where would we like the student’s skills to be at the end of the designated intervention period? • Progress Monitoring: Is the student developing the skills we are teaching? • Decision Rules: Do we need to change what we are doing?

10. Baseline Data: Comparison to Peers How is Baseline Data Collected? • Student is given the grade level probes used in norming • Administration starts with the first item on the page each time • Over 3 days in one week • Median scores in each probe area are compared to the grade level norms

11. Determining Where to Intervene • Skills of student compared to peers • Lowest level skills (the building blocks)

12. Back Sampling and Diagnostic Assessment • Back Sampling • If the student’s skills in the lowest skill areas on grade level are weak, then baseline data is collected on the probes for the previous grade level to inform instruction. • Diagnostic Assessment • Can be completed by reviewing actual skills in an area. Ex. Which sounds/blends/digraphs does the student know? Which math facts are known?

13. Skills: Discrete or Complex • Discrete Skills are generally taught by practice and over learning. These might include as phonemic awareness, blends, sight recognition, fluency, math calculation, etc. • Complex Skill Sets required a combination of skills to perform. These might include fluency, vocabulary, reading comprehension, math concepts, written expression, etc

14. Fluency: Discrete or Complex? • Fluency might be considered • Complex if it is the highest level skill on which the staff are intervening with the student because the student is still working on building blocks or prerequisite skills such as phonological awareness, phonics and/or sight words • Discrete if it is the lowest level skill on which the staff are intervening because the requisite skills are there, but the student needs practice for speed

15. Plotting Baseline Data • Student A--0, 5, 3 • What is the median score? • Where do you plot it on the graph? • Student B—25, 38, 40 What is the median score? • What is the median score? • Where do you plot it on the graph? • Student C—0, 0, 10 • What is the median score? • Where do you plot it on the graph?

19. Goals: What should we use? Where would we like the student’s skills to be at the end of the designated intervention period? Considerations: • Is the skill a building block to higher level skills? • Is the skill discrete or complex? • Is it likely to take more or less time and/or intensity to bring the skills up to proficiency?

20. Types of Goals • District (NHCS) Norms • District Behavior Standards • Growth Rates • NHCS Data - Can be done for any skill, any grade level for which norm data is available • Realistic/Ambitious (Fuchs and Fuchs)-Fluency Only • Class Norms • Minimum Celeration Finder

21. 1. Using District Norms • Goals are selected from the norms tables by grade level. The norms tables are located in the Special Education and Related Services manual on the web. • Often the 25th percentile is chosen, but there are times when that is too ambitious for a student and the 13th is used instead. • Goals be selected by the date of the end point of the intervention. Example: If the baseline data was from the week of 10/26/09 (Fall Norms) but the goal is set for 6 weeks later (12/7/09), the goal should be selected from the Winter Norms.

22. 2. NHCS Behavior Standards: Will be addressed later in the presentation

23. 3. Growth Rate: Using Growth Rate to Set a Goal • NHCS • Determine the number of weeks of intervention. • Multiply growth rate from chart by number of weeks of intervention • Add to baseline median • Plot goal at number of weeks on the chart.

24. 3. Growth Rate: How Do We Calculate It? • Take a seasonal norm (fall to winter or winter to spring) • Subtract the earlier seasonal norm, and divide by 10 (for NHCS probes) or by 18 for DIBELS • For example; Winter Norm minus Fall Norm, divided by ten weeks

25. 3. Growth rate calculation: Fall to Winter • CALCULATE: • First grade, ID words in sentences • Winter Norm minus Fall Norm, divided by ten weeks • 76.97- 48.15 / 10 = 2.82 words per week • USE GROWTH RATE TO SET GOAL • For this reading fluency measure, students are acquiring a little under 3 words per week, thus if a intervention plan is for six weeks then the student should acquire a little over 18 words in the six weeks • Add 18 to baseline median to set goal.

26. 3. Growth Rate Calculations: Fall to Spring • CALCULATE: • First grade, ID words in sentences • Spring Norm minus Fall Norm, divided by 20 weeks • 99.07- 48.15 / 20 = 2.54 words per week • USE GROWTH RATE TO SET GOAL • For this reading fluency measure, students are acquiring about 2.5 words per week, thus if a intervention plan is for six weeks then the student should acquire a little over 15 words in the six weeks • Add 15 to Baseline Median to set the goal.

27. 3. Using Growth Rates • Realistic/Ambitious Growth Rates for Fluency (Fuchs and Fuchs) • The process is the same as with NHCS Growth Rates

28. Realistic Growth Rates Gr 1 2 words/week Gr 2 1.5 wrds/week Gr 3 1 words/week Gr 4 .9 words/week Gr 5 .5 words/week Ambitious Growth Rates Gr 1 3 words/week Gr 2 2 words/week Gr 3 1.5 wrds/week Gr 4 1.1 wrds/week Gr 5 .8 words/week Realistic/Ambitious Growth Rates (Fuchs and Fuchs)--Fluency

29. 4. Classroom Norms: Why? • You may choose to norm a class or grade level on a probe that was not used for state or local norming. Most likely when probes from previous grade levels need to be used to determine entitlement. • To do class or grade level norms, the school must: • Give the probe(s) to the entire group, • Administer each probe 3 times in a week. • You will probably want to do this Fall, Winter and Spring.

30. 4. Classroom Norms: How do you get one? • To do class or grade level norms, the school must: • Give the chosen probe(s) to a sample that is representative of the school population • In a school with heterogeneous grouping, the student’s classroom will do • In a school with homogeneous grouping you may have to balance out the student’s classroom with another or even norm the entire grade • Administer each probe 3 times in a week.

31. 4. Classroom Norms: How do you get one? • Once you have the class or grade level data • Add all the students’ scores for all 3 days together. • Compute the Mean score by dividing by the total of the students’ scores by the total number of probe administrations. • Ex. 3 administrations with 25 students would equal a divisor of 75. • You may want to do a mean for more than one season: Fall and Winter, Winter and Spring

32. 4. Using Classroom Norms • Setting Goals: • Class or Grade Level Mean • Minimum Celeration Finder • Determining Whether a Student’s Skills are Discrepant After Intervention: • Divide the class or grade level Mean by the Median of the Student’s last 3 data points. • Class Mean ÷ Student’s Median ≥ 2

33. Computing Two Times Discrepant Ex. Class Mean is 40. Student Median is 20. 40 ÷ 20 = 2 Student is discrepant. Non Ex. Class Mean is 40. Student Median is 22. 40 ÷ 22 = 1.89 Student is not discrepant.

34. 5. Minimum Celeration Finder • Celeration rates from the Precision Teaching program can be used for goal setting consideration • The lines on the overlay indicate different celeration slopes or rates of progress • Depending on how ambitious the team wants the goal to be different rates of progress can be used

35. Minimum Celeration

36. Plotting Goals and Aimlines • Student A--0, 5, 3 (Grade 2, Sight Words, Fall to Winter) • District Norm • NHCS Growth Rate • Example uses 13th Percentile growth rate • May use the growth rate at 25th or 50th percentile

39. Plotting Goals and Aimlines • Student B—25, 38, 40 (Grade 2, Words in Passage, Winter to Spring) • District Norm • Realistic/Ambitious Growth Rate

42. Progress Monitoring: Why? • Time and cost efficient • Sensitive to changes over short periods of time • Frequent and repeated data collection (dependent upon student’s level/tier) and analysis of student performance • Use data to inform instruction in specific skills.

43. Progress Monitoring • How does it differ from collection of baseline data? • Use alternative probes with the baseline probe included as the cycle of alternative probes allows. • For probes such as phonemes, blends, sight words, or math calculation, start at the line after the last line used on the third baseline probe. • Alternate probes for reading fluency (Skill Builders and DIBELS); for reading comprehension (Blue binder books by grade level from Fuchs and Fuchs) and Math Concepts (Black line Masters—Orange binder book)

44. Progress Monitoring: When? • Tiers 1 and 2: • May use DIBELS, ClassScapes, Reading or Math Assessment, Probes, etc. • Generally less frequent or even pre-intervention and post-intervention.

45. Progress Monitoring: When? • Tier 3: • Grade Level Probes Normed by the County, • DIBELS K-2 (Local Norms), and/or Probes normed in the classroom or grade level. • At least two probe areas • Must be given a minimum of 5 times in 2 weeks.

46. Type of Progress Monitoring Tier 3 Normed Probes in 2 Areas: NHCS Tier 2 Depends on Intervention Plan Evaluation Design Tier 1 Depends on Intervention Plan Evaluation Design

47. Frequency of Progress Monitoring Tier 3 Every Other Day NHCS Tier 2 Depends on Intervention Plan Evaluation Design Tier 1 Depends on Intervention Plan Evaluation Design

48. REVISIONS OF HYPOTHESES/INTERVENTIONS How do we know when to change the intervention?

49. Decision-making rules: What type of skill is involved? • Discrete Skills such as phonemic awareness, blends, sight recognition, fluency, math calculation: • Consider 3 to 4 below the line to change hypothesis/intervention • 4 to 6 above the line to raise goal and aimline or discontinue (25th percentile) and move on the next skill in the hierarchy

50. Decision-making rules: What type of skill is involved? • Complex Skill Sets such as fluency, vocabulary, reading comprehension, math concepts, written expression, etc: • Consider how many days/weeks it might take to show improvement or be certain skill is developed Given probe administration 5 times in two weeks: • How many data points below the line should be considered before changing hypothesis/intervention? • How many data points above the line should be considered before changing the goal or considering a change back to Tier 1 or 2?