CBM, Charting, Progress Monitoring, and Data Based Instructional Decision Making in a RTI Model

1. CBM, Charting, Progress Monitoring, and Data Based Instructional Decision Making in a RTI Model Tom Jenkins, Ed.D. Educational Consultation Services, LLC Wilmington, NC

2. What you can expect today • CBM, CBA, CBE • Instruction on progress monitoring • Examples and hands-on activity • Final thoughts and conclusions

3. What is CBM? • Curriculum-based measurement • Data collection tools derived directly from the curriculum that student is expected to learn

4. CBM • CBM is believed to reduce the gap between assessment and instruction • Aides teachers in generating superior student achievement • Improved communication • Higher level of sensitivity • Enhancement of the database • Administration time is shorter • More cost effective

5. CBM • The simplicity of CBM allows for quick and easy peer referencing • Normative data can be collected • This allows for comparison of a student’s performance to his/her actual peer group • More representative geographically, culturally, ethnically, and has been exposed to similar instructional environment

6. CBM • CBM has been shown to posses high levels of reliability • 42 one-minute CBM type assessments in reading, math, and written expression for grade K-5 were found to have reliability coefficients between .90-.99 with just three one-minute administrations (Jenkins, 2002)

7. CBM • Discriminant Validity • Several studies have demonstrated the ability of CBM to differentiate between students receiving special education services, students receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987).

8. CBM Procedures • Remember different modalities • Given the nature of reading probes they must be administered individually • Math and written expression may be administered in a group setting

9. CBM Procedures • Scoring • One minute administration time except for written expression • Number ID is scored corrects for minute; however, math computations are scored digits correct per minute • Written expression is scored according to correct sequences

10. Example • Administration example • Work in pairs • Each person gets a chance to adminster

11. CBM Procedures • Written expression examples

12. Progress Monitoring • Basic Principle #8 of PSM • Progress monitoring an essential aspect of the intervention phase • Basic Principle #9 of PSM • Decision making in regards to the effectiveness of an intervention is based on analysis of progress monitoring data in relation of goal

13. Progress Monitoring • Essential for four reasons • There is no guarantee that interventions will be successful, thus the intervention must be “tested” to evaluate effectiveness • Increased emphasis of specific outcomes for students, data base must be generated to guide intervention decision making • Pre/post testing has be shown to be unreliable (small amount of data) and provides too little data to allow for instructional decision making – progress monitoring allows for evaluation of level of performance and rate of learning • Research has shown that progress monitoring is associated with improved educational outcomes

14. Progress Monitoring • Definition • Frequent and repeated data collection (no less than 2-3 times a week) and analysis of student performance • Data is collected during intervention and provides basis for intervention effectiveness

15. Progress Monitoring • Essential components that must be in place for successful progress monitoring • A well-defined behavior • A measurement strategy • Identification of student’s current level of performance (baseline) • Intervention • Goal • Graph • Decision-making plan

16. Well Defined Behavior • Target behavior, observable, measurable, and specific • Focus on enabling skills • Skills that are prerequisite skills for more complex skills • Deficiencies in enabling skills often adversely affects performance on global assessments

17. Well Defined Behavior • Enabling skills for reading • Phonemic awareness • Alphabetic understanding • Fluency • Sight words • Comprehension

18. Well Defined Behavior • Enabling skills for math • Number sense • Facts • Computation • Applications • Problem solving • Enabling skills for written expression • Mechanics • Expression

19. Well Defined Behavior • Enabling skills for behavior • Social skills • Work completion • Compliance • Problem solving skills

20. Measurement Strategy • Frequent and repeated collection of data • Time and cost efficient • Sensitive to changes over short periods of time

21. Current Level of Performance (Baseline) • Gathered prior to intervention • Repeatable • Provides comparison for progress data • Helps set goal • Median score – why?

22. Intervention • Match intervention to problem • Humans tend to employ interventions with which they are comfortable instead of intervention that the student needs • Intervention should be developed with the expectation that it will be altered in some way as a result of the progress monitoring data • No intervention works all of the time for every student

23. Goal • Standard against which progress can be compared • Allows for aimline to be established • Possible goals • Norms • Percentile cutoff • Realistic growth rates • Ambitious growth rates • Minimum celeration • Local growth rates

24. Computing Growth Rates • Winter Norm minus Fall Norm, divided by number of weeks between norming projects • Gives you a growth expectancy for each week of school year • Allows for obtaining student’s baseline then monitoring progress while comparing to growth expectancy

25. Computing Growth Rates • Example – first grade, ID words in sentences • Winter Norm minus Fall Norm, divided by number of weeks between norming projects • 67.9119-35.2694 / 10 = 3.2643 words per week

26. Realistic Growth Rates Gr 1 2 words/week Gr 2 1.5 wrds/week Gr 3 1 words/week Gr 4 .9 words/week Gr 5 .5 words/week Ambitious Growth Rates Gr 1 3 words/week Gr 2 2 words/week Gr 3 1.5 wrds/week Gr 4 1.1 wrds/week Gr 5 .8 words/week Growth Rates based on research by Doug and Lynn Fuchs

27. Goal • 1.25 minimum celeration, Precision Teaching

28. Graph • Provides a visual representation of a large amount of data • A visual representation of student’s acquisition of skills and allows for easier analysis of progress • Semi-log chart • Equal interval charts can misrepresent data, depending on how axis is quantified • Equal interval charts assumes equal amounts of progress between all data points • Precision Teaching

29. Decision Making Plan • Facilitates interpretation of data • Should include • Rule for raising performance goal – 6/4 consecutive data points above the aimline • Rule for altering the intervention due to lack of progress – 3/4 consecutive data points below the aimline • Deno/Allison

30. Decision Making Plan • If the decision is to adjust an intervention, small changes or refinements are recommended before major changes • However, changes should be substantial enough that it has a possibility to result in improved student performance • If making an adjustment, do not make two at the same time. It may result in the team being unable to determine what caused increased student performance

31. Decision Making Plan • As student performance approaches goal, team must decide • Raise the goal • Begin work on another target behavior • Discontinue PSM process • If in level four consider • Discontinuation of EC services

32. Charting Activity • A means of interpreting large amounts of data • Allows for relating performance trends to desired performance • Research has shown the charting also facilitates positive student outcomes (Fuchs, 1989)

33. Charting Activity • Definition • Visual depiction of the student’s performance data, relative to the goal and aimline • Includes baseline data, goal, aimline, and progress monitoring data

34. Charting Activity • Procedures • Write the prediction/goal statement • Depict baseline data collection phase on the chart, indicate the median score, identify with a heavy dark line • Depict the goal at the end of the anticipated intervention phase • Depict the aimline by connecting the baseline median with the goal, this gives you expected rate of progress

35. Charting Activity • Activity • Plot the baseline data • Monday – 14 • Tuesday – 10 • Wednesday – 6 • Thursday – 5 • Friday – 10 • Plot the median and signify with a heavy dark line

36. Charting Activity • Plot the goal at the end of eight weeks • 40 • Indicate the aimline

37. Charting Activity • Performance trends should be analyzed periodically • Trend above aimline – raise the goal • Trend below aimline – adjust intervention • If changes are made to the intervention, indicate change on the graph with a squiggle line • Describe the changes on the back of the chart • This allows for understanding of specific instructional adjustments that were successful/unsuccessful

38. Charting Activity • Plot the first two weeks progress monitoring data • Week One • Tuesday – 16 • Thursday – 14 • Week Two • Monday – 10 • Wednesday – 18 • Friday – 16 • Make an informed decision regarding the effectiveness of the intervention

39. Charting Activity • Remember • Rule for raising performance goal – 6/4 consecutive data points above the aimline • Rule for altering the intervention due to lack of progress – 3/4 consecutive data points below the aimline • Deno/Allison

40. Charting Activity • Plot progress monitoring data for next two weeks • Week Three • Tuesday – 14 • Thursday - 10 • Week Four • Monday – 15 • Wednesday – 16 • Friday – 16 • Make an informed decision regarding the effectiveness of the intervention

41. Charting Activity • Phase two or three of the intervention? • Plot the progress monitoring of the next two weeks • Week Five • Tuesday – 24 • Thursday - 26 • Week Six • Monday – 26 • Wednesday – 24 • Friday – 26 • Make an informed decision regarding the effectiveness of the intervention

42. Charting Activity • Phase two or three of the intervention? • Plot the progress monitoring of the next two weeks • Week Seven • Tuesday – 28 • Thursday - 30 • Week Eight • Monday – 32 • Wednesday – 36 • Friday – 38 • Make an informed decision regarding the effectiveness of the intervention • Final decision?

43. Charting Activity • More information than you wanted to know • Trend lines • Line that you draw through a series of data points that represents the student’s actual rate of progress • If trend line slope is flatter than aimline slope, then adjust intervention • If trend line slope is steeper than aimline slope, then adjust goal • If slopes are the same, make no change

44. Charting Activity • Trend line procedures • Draw vertical line in middle of graph, half data points on one side, half on the other – if odd number of data points, put line through middle point • Draw a vertical line in first half of data to separate data half and half • Draw a vertical line in second half of data to separate data half and half

45. Charting Activity • Trend line procedures • Draw a horizontal line through median of first half of data – form an intersection with vertical line • Draw a horizontal line through median of second half of data – form an intersection with vertical line • Connect the two intersections • Make your decision, is it the same decision?

46. Thoughts and Conclusions • What if we took it a step further? • What if we used the intervention/progress monitoring/charting process to move students towards a CBM reading fluency score that we knew predicted success on EOG’s and EOC’s?

47. Thoughts and Conclusions • Espin et al, 2002 found correlation between CBM reading fluency scores and Minnesota grade 8 state standards test scores to be .78 • Grade 5 correlation between CBM reading fluency scores and Minnesota Comprehensive Assessments - .77

49. Thoughts and Conclusions • Similar results were found in Minnesota for grades 3 and 5 • Similar results have been found in different states