Using Curriculum-Based Measurement to Evaluate Student Writing and Plan Instruction

1. Using Curriculum-Based Measurement to Evaluate Student Writing and Plan Instruction Stacy L. Weiss Josh Bock Casey O’Connell Indiana University-Bloomington

2. Agenda • Writing in today’s world • Assessment – Instruction interaction • Basics of Curriculum-Based Measurement • Written Expression CBM • Scoring procedures • Practice • Tips for CBM • Writing interventions

3. Writing Today • Written expression is key in a digital world (Yancey, 2009) • Writing skills are essential for well-paying professions (National Commission on Writing, 2004) • Written expression is needed to demonstrate learning across subject areas

4. Writing Today • Written expression is a multi-faceted process (Berninger, Garcia, & Abbott, 2009) • A large percentage of students with disabilities have difficulty with basic writing skills (NAEP, 2007) • Writing needs to have a greater focus in today’s schools

5. Teaching Cycle

6. Curriculum-Based Measurement Short, simple, efficient assessments

7. Purpose of CBM • Formative assessment • Class-wide screening • Progress monitoring • Goal setting • Program evaluation

8. Administration of Written Expression CBM • Standardized directions • Students respond to a story starter • Animals can’t talk, but… • Just when we got into our seats, the… • Yesterday, I went to the park and… • I was sleeping soundly when…. • I opened the front door very carefully when… http://www.aimsweb.com/measures-2/written-expression-cbm/

9. Administration of Written Expression CBM • Students have: • 1 minute for planning • 3 or 5 minutes for writing • Students write a first draft

10. TWW Scoring Guidelines • A word is any group of letters, even if the word is misspelled • Abbreviations count as a word • Numerals do not count as words (e.g., 5), unless they are in a date (e.g., Nov. 4) or after a dollar sign ($4.00). Numbers that are spelled out count as words (e.g., five) • A repeated story starter is counted as words written

11. WSC Scoring Guidelines • Proper nouns must be capitalized to be spelled correctly (e.g., mary is incorrect) • Words written as proper nouns or names should be given the benefit of the doubt (e.g., Marey) • The first word of a sentence does not need capitalized to be a correctly spelled word (the girl went outside = 4 correctly spelled words) • Common abbreviations are spelled correctly (e.g., TV) • Words with reversed letters are spelled correctly, unless the reversal makes another letter that makes the word misspelled.

12. Correct Word Sequences • Number of pairs of words that include the correct grammar and spelling • Generally good indicator of writing skills overall • For older students, calculate the number of correct word sequences minus incorrect word sequences (CWS – IWS) for accuracy

13. CWS Scoring Guidelines • Capitalization, spelling, and ending punctuation must be correct • Correct sequences are grammatically correct (e.g., My mom and I are…) • Homophones need to be correct (e.g., two/to/too) • Numerals and symbols are not included in the scoring

14. Evaluate Results • Examine writing samples for patterns of errors • Spelling errors • Grammatical errors • Handwriting difficulties • Letter reversals • In addition to the raw score, the percentage of correct responses can be calculated.

15. Progress Monitoring: Expected Growth • Compared to reading CBMs of oral reading fluency, limited research on expected progress • Norms are available by percentile of progress between three benchmarks for Correct Word Sequences (Hosp, Hosp, & Howell) • After a short writing intervention, students wrote an average of three more words (TWW) (Gansle et al., 2004)

16. Progress Monitoring: Goals • Using norms, goals can be written for CWS • For IEP goals, use a combination of fluency and accuracy • Example Goal: In response to a story starter, Rita will produce 39 correct word sequences in three minutes with 90% accuracy. Hosp, Hosp, & Howell, 2007

17. Progress Monitoring: Graphing

18. Plan and Deliver Instruction Writing Process, Grammar, and Spelling

19. Self-regulated Strategy Development • Developing and activating background knowledge • Discussing the strategy • Cognitive modeling of the strategy • Memorization of the strategy • Supporting the strategy • Independent performance

20. POW • Pick my idea • Organize my notes • Write and say more Reid & Leinemann, 2007

21. WWW What=2, How=2 • Planning strategy for narrative writing • Questions • Who is the main character? • Who else is in the story? • When does the story take place? • What does the main character do? • What happens when he/she tries to do it? • How does the story end? • How does the main character feel? Reid & Leinemann, 2007

22. Graphic Organizers http://www.eduplace.com/graphicorganizer/

23. Writing Strategy - COPS • An easy-to-use revision strategy • Assists students in making mechanical revisions • Students are prompted with questions to help them review any revisions that need to be made • The mnemonic of COPS helps student remember the steps that need to be followed (Reid & Lienemann, 2007)

24. CHave I capitalized the first word ? OHow is the overall appearance? P Have I put in commas and punctuation? SHave I spelled all the words right?

25. Spelling Strategies • Cover, Copy, Compare (Erion et al., 2007) • Look at the spelling word • Cover the word • Copy the word from memory • Uncover word and Compare with model • Direct, explicit instruction in spelling patterns (Wanzek et al., 2006)

26. Suggestions for Incorporating CBM • Be organized • Make a schedule • Take breaks when scoring • Reward yourself • Collaborate with others

27. Contacts Stacy L. WeissAssistant Professor in Special Education(812) 856-8139stweiss@indiana.edu Josh Bockjbbock@indiana.edu Casey O’Connellcatoconn@indiana.edu

28. Resources Mather, N., Wendling, B. J., & Roberts, R. (2009). Writing assessment and instruction for students with learning disabilities. San Francisco: Jossey-Bass. ISBN-13: 978-0470230794. Riley-Tillman, T. C., & Burns, M. K. (2009). Evaluating educational interventions: Single case design for measuring response to intervention. New York: The Guilford Press. ISBN-13: 978-1606231067. Troia, G. A. (Ed). Instruction and assessment for struggling writers: Evidence-based practices. New York: Guilford Press. ISBN-13: 978-1593859923.National Center on Response to Interventionhttp://www.rti4success.org/index.php Intervention Central Curriculum-Based Measurement Warehousehttp://www.interventioncentral.org/index.php/cbm-warehouse#writtenExpression National Center on Student Progress Monitoringhttp://www.studentprogress.org/

29. Works Cited Berninger, V. W., Garcia, N. P., & Abbott, R. D. (2009). Multiple processes that matter in writing instruction and assessment. In G. A. Troia’s (Ed.). Instruction and assessment for struggling writers: Evidence-based practices (pp. 15-50). New York: The Guilford Press. Erion, J., Davenport, C., Rodaz, N., Scholl, B., & Hardy, J. (2009). Cover-copy-compare and spelling: One verses three repetitions. Journal for Behavioral Education, 18, 319-330. Gansle, K. A., Noell, G. H., VanDerHeyden, A. M., Slider, N. J., Hoffpauir, L. D., Whitmarsh, E. L., Naquin, G. M. (2004). An examination of the criterion validity and sensitivity to brief intervention of alternative curriculum-based measures of writing skill. Psychology in the Schools, 41, 291-300. Hosp, M. K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBMs: A practical guide to curriculum-based measurement. New York: The Guilford Press. National Commission on Writing (2004, September). Writing: A ticket to work or a ticket out. A Survey of Business Leaders. The College Board. Retrieved from: http://www.host-collegeboard.com/advocacy/writing/index.html Reid, R., & Lienemann, T. O. (2006). Strategy instruction for students with learning disabilities. New York: The Guilford Press. Wanzek, J., Vaughn, S., Wexler, J., Swanson, E. A., Edmonds, M., & Kim, A. (2006). A synthesis of spelling and reading interventions and their effects on spelling outcomes for students with LD. Journal of Learning Disabilities, 39, 528-543. Yancey, K. B. (2009). Writing in the 21st Century: A report from the National Council of Teachers of English. Urbana, IL: National Council of the Teachers of English. Retrieved from http://www.ncte.org