ABCs of CBMs

1. ABCs of CBMs Summary of A Practical Guide to Curriculum-Based Measurement By: Michelle Hosp, John Hosp, & Kenneth Howell

2. What will CBM provide? • CBM provides functional information required to inform educational decision making, including: • Goal setting • Planning • Monitoring

3. What is CBM • CBM is composed of a set of standard directions, a timing device, a set of materials, scoring rules, standards for judging performance, and record forms or charts.

4. Attributes of CBM • Alignment • Tested on the curriculum • Technically adequate • Established reliability and validity • Used to summarize both a student’s level of performance and rate of progress • Makes use of criterion-referenced measures

5. Attributes of CBM cont. • Standard procedures are used • Standard tasks are used for each content • Standard procedures are followed for selecting or constructing testing materials • Standard administration and scoring directions exist for each procedure 5. Performance sampling is used (producing what is sometimes called behavioral or performance data)

6. Attributes of CBM cont. • Decision rules are put in place. These rules are based on performance criteria and are standardized through sampling or experimental procedures. • Emphasized repeated measurement over time. These repeated measures illustrate the degree to which an instructional intervention is producing learning.

7. Attributes of CBM cont. • Efficient. The measures can be given in a short period of time and quickly. • Data can be summarized efficiently. The results are summarized and interpreted as simple behavioral/performance statements.

8. How is CBM different? CBM focuses on the curriculum and was designed to function within a problem-solving paradigm (e.g. Responsiveness to Instruction RtI)

9. Differences? • Alterable Variables - Designed to fill the need for a data collection system that would produce the information required to guide instruction. • Low-inference Measure - CBM was not developed to explain how learning does or doesn’t occur, and it was not designed to conform to any particular theory about how students think, attend, remember, or process information. Therefore, inference and conjecture about what the resulting scores mean is kept to a minimum.

10. Differences? • Criterion-references Measures - Used to determine if students can demonstrate their knowledge by reaching specified performance levels on certain tasks. CBM came directly out of an intervention program. This means the measures must be: • Aligned with curriculum • Sensitive to instruction • Repeatable so that progress monitoring could occur • Able to determine if student has mastered task

11. How does CBM relate to RtI? Key components of any good RtI approach are the use of screening/benchmarking and progress-monitoring assessments. CBMs are an efficient tool for meeting these components.

12. More about CBM • CBM is not an instructional method or intervention • CBM is not a curriculum • The defining elements of CBM are the curriculum-based procedures for designing, administering, and scoring measures and for recording, summarizing, and interpreting the data that result from those measures. • A measure gets to be a CBM instrument if it is designed, administered, and scored according to established CBM procedures.

13. Different types of CBM • General Outcome Measures (GOMs) • Used to sample performance across several goals at the same time by using capstone tasks that are complex in the sense that they can only be accomplished by successfully applying a number of contributing skills. • Advantages • Efficient by measuring several subskills • Recognizes the limitations of isolating subskills from the context in which they normally are expected to function • Visual displays of progress will show longer acquisition slopes, allowing adequate opportunities for progress monitoring and data-based instructional modifications • Disadvantage • May need more diagnostic information • Example - Oral Reading Fluency

14. Different types of CBM cont. • Skills-based Measures • Designed to accomplish many of the functions of GOMs • Advantages • Will produce long acquisition slopes like GOMs • Can yield some analytical information as long as steps are taken to ensure that an adequate sample of each kind of item is provided and that the items are cross-referenced to goals. • Disadvantage • At any given time only a few items on the test will be directly related to what the student is currently learning. • Example - Math Computation

15. Different types of CBMs cont. • Mastery Measures • Used on parts of the curriculum that contain discrete and easily identified sets (or domains) of items that are closely related by some common skill, theme concept, or solution strategy. • Advantages • Focuses on a particular set of skills. • Can help troubleshoot a problem and need to do specific-level testing • Monitor learning when a skill is being taught in isolation • Disadvantage • Not good for surveying general levels of performance or for monitoring growth on long-term goals. • Example - Math Facts

16. CBMs & the Curriculum • What is the definition of curriculum? • Curriculum is “what you teach” not “how you teach.” • Therefore, you do not need to have assessments using the same formats and examples as the instructional materials.

17. Reasons for this position • Instructional programs do not follow the same sequences and schedules. • Program-specific tests may not tell you if the learning has generalized. (You do not want a student who can only work problems that are presented in a certain format). • Program-specific tests will make the teacher dependent on the program.

18. CBMs can be used for… • Screening/benchmarking decisions • Progress-monitoring decisions • Diagnostic decisions • Outcome decisions • CBM is most appropriate for screening/benchmarking decisions and progress-monitoring decisions.

19. Where do CBMs get the performance standards? • Exemplar Sampling - Draw a sample of successful students for the skill in question. • Norm Sampling - Testing a large group of students and using the average scores at a grade level or some percentage of them as the target level of performance for that grade. • Predictive Validity or Benchmarks - Involves empirically or statistically determining the level of performance that reliably predicts successful performance on a different outcome measure, usually at a future date. • AIMSweb uses norm sampling and benchmarks.

20. CBM examples

21. Reading CBM (Oral Reading Fluency) • Reading CBM provides a reliable and valid way to: • Identify students who are at risk for reading failure • Identify which students are not making adequate progress given the instruction they are receiving • Identify students’ instructional level • Identify which students need additional diagnostic assessment

22. Reading CBM (Oral Reading Fluency) • Automaticity is important because it demonstrates the student has mastered the skill. • In order to read a passage of text aloud quickly and accurately, you need to use a variety of different literacy skills, including decoding, vocabulary, and comprehension (particularly accessing prior knowledge). • This is what makes ORF a good predictor of future reading performance.

23. FAQ about Oral Reading Fluency • Should I have students read the title on the page or should I it to them? • You should not have the student read the title nor should you read it to the student. It is not counted in the total words for the passage and may give the students some background information. Remember, this is a time to assess, not teach. • Can the benchmark scores be used to put students in instructional groups? • Yes, if you have students with similar instructional needs. These groups should be flexible, and students should be evaluated and regrouped every 6-8 weeks. • Other questions can be found in the book.

24. MAZE • A measure that looks at comprehension.

25. Early Literacy • These measures require the student to be accurate as well as fluent allowing one to determine how automatic students are at performing a task. • Remember, automaticity is important because it represents a higher level of skill mastery.

26. Early Literacy Measures • Initial sound fluency and phoneme segmentation fluency are the two measures designed to assess phonological awareness. • Nonsense word fluency is a measure designed to assess decoding skills. • Letter naming fluency is used as a risk indicator for reading. • Word identification fluency is a measure of phonics and fluency.

27. Spelling • Spelling is an important skill for writing and evaluating that also provides information about a student’s decoding. • Good spellers are always good readers, but the reverse is not always true; a poor speller can be a good reader or a poor reader.

28. FAQ about Spelling CBM • Can I give students the list prior to the assessment? • No. This assessment is a general measure of students’ skill at applying the spelling rules they have been previously taught. Giving students the list prior to the assessment increases the chance that students will attempt to memorize the words instead of learning the spelling skills being taught. • The list for this week doesn’t include the spelling rules I just taught. Can I choose one that does? • No. Avoid teaching to the test and remember Spelling CBM is designed to assess general, not specific, skill mastery. • Should I teach students the spelling rules they miss on the spelling lists? • Yes. Whenever students are missing previously taught rules, it is beneficial to reteach the material. This may be an indication that students did not fully understand the concept. Teachers should, however, continue to use the scope and sequence presented in the curriculum. • Other questions can be found in the book.

29. Writing • Several components of written communication pertinent to student progress in written language include: • Writing fluency, syntactic maturity, vocabulary or semantic maturity, content, and conventions. • Numerous studies have presented CBMs available for many of these target areas, including: • Total words written, words spelled correctly, correct writing sequences, and total correct punctuation.

30. Math • Reason for using a math CBM • Most math assessments do not provide information about fluency, which provides information about skill mastery, but CBM does. • Single-skill math sheets are not appropriate for screening or progress monitoring (because it only assesses one skill), but it can be used to gain some diagnostic information or to use as a starting point for a curriculum-based evaluation approach to decision making. • When scoring Math CBM, the number of correct digits in the solution to the problem is used because it is a more sensitive measure to change.

31. Charting and Graphing CBM Data to Help Make Decisions

32. Three methods for setting goals • End-of-Year Benchmarks • This is the lowest score we would accept that would indicate a student is not at risk for future academic failure.

33. Three methods for setting goals, cont. • Norms • When available, national norms are good to use. • Come in two forms: • Levels of Performance: Use same process as with end-of-year benchmarks. The difference is that these are based on typical performance of same-grade peers rather than a criterion for proficiency that predicts performance on outcome measures. • Rate of growth or progress: This can be calculated as an average weekly gain.

34. Three methods for setting goals, cont. • Intraindividual Framework • Uses the student’s current level of performance and rate of progress to set end-of-the-year goals. • It may underestimate a student’s rate of learning and may never catch the student up if the student started behind.

35. Graphing • To graph, identify a specific target to work toward and number of weeks it will take to get there.

36. Decision Rules • Two methods are used for making decisions: • Data point analysis - The data points on the graph for each week are examined. • Trend line analysis - Represents the student’s observed rate of progress, which can be compared to the expected rate of progress as indicated by the goal line.

37. How to use CBM in RtI • The same CBM measures can be used with all students across the various tiers of instruction. • Types of decisions • Effectiveness of the instructional program that a student is receiving. CBM does not give you information specific enough to determine which instructional approach to use or how to alter the current one - this is a time for teachers’ professional judgment. • Eligibility for special education. The most common approach of eligibility is called the dual discrepancy method. CBM provides you with performance data and progress data. You can compare both of these types of data to standards.

38. How to learn more • You can learn more about CBMs by reading the book, The ABCs of CBM. • Reviewing the website: National center on Student Progress Monitoring: www.studentprogress.org