1 / 106

Content Prepared by Roland H. Good III, University of Oregon

System-wide Review in a Comprehensive Reading First Assessment Plan Presentation based on the presentation of the same title at the 2005 National Reading First Conference. Content Prepared by Roland H. Good III, University of Oregon.

apu
Download Presentation

Content Prepared by Roland H. Good III, University of Oregon

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. System-wide Review in a Comprehensive Reading First Assessment PlanPresentation based on the presentation of the same title at the 2005 National Reading First Conference Content Prepared by Roland H. Good III,University of Oregon http://darkwing.uoregon.edu/~rhgood/salt_lake_city_handout.pdf Salt Lake City, Utah

  2. The Purpose of Assessment is to Change Life Trajectories for Children In this presentation, the purpose will be to address the following issues and questions about Reading First Assessment: • A comprehensive Reading First assessment plan incorporates screening, progress monitoring, diagnostic, and outcome assessment in an integrated educational decision-making model. • A comprehensive assessment plan is needed to blend these purposes together in a way that can be maintained over time with the resources available, and in a way that is not so time-consuming that it compromises the instructional mission of the school. Salt Lake City, Utah

  3. Beginning Reading Core Areas #1. Phonemic Awareness: The ability to hear and manipulate sound in words. #2. Phonics: The ability to associate sounds with letters and use these sounds to read words. #3. Fluency: The effortless, automatic ability to read words in isolation (orthographic reading) and connected text. #4. Vocabulary Development: The ability to understand (receptive) and use (expressive) words to acquire and convey meaning. #5. Reading Comprehension: The complex cognitive process involving the intentional interaction between reader and text to extract meaning. Salt Lake City, Utah

  4. Model of Big Ideas, Indicators, and Timeline Adapted from Good, R. H., Simmons, D. C., & Kame'enui, E. J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5, 257-288. Salt Lake City, Utah

  5. Alphabetic Principle Goal Instructional Goal: To have a powerful strategy to encounter an unknown word and confidently obtain a reasonable pronunciation of the word. NWF Indicator: 50 correct letter sounds with 15 words recoded by middle of first grade. • Two reasons for students to have the alphabetic principle: 1. Beginning readers encounter a lot of new words. A powerful strategy to attack them is valuable. 2. The alphabetic principle establishes the pathways and mechanisms by which a skilled, mature, fluent, confident reader accesses words and word meanings. Automaticity with the code is based on phonics/phonological awareness processes. Salt Lake City, Utah

  6. DIBELS Nonsense Word Fluency (NWF) Here are some more make-believe words(point to the student probe).Start here(point to the first word)and go across the page(point across the page).When I say, “begin”, read the words the best you can. Point to each letter and tell me the sound or read the whole word. Read the words the best you can. Put your finger on the first word. Ready, begin. Salt Lake City, Utah

  7. DIBELS are Indicators • DIBELS is a toothpick • DIBELS are not designed to provide an exhaustive assessment • DIBELS provides an efficient indicator of essential literacy skills acquisition like a toothpick provides an efficient way to tell if the cake is baked. • If the toothpick has dough on it, what should we do? Salt Lake City, Utah

  8. Bake the Whole Cake, Don’t Just Cook One Place! • Using a torch to cook only the place we checked with the toothpick would not produce a very satisfying cake! Salt Lake City, Utah

  9. Alphabetic Principle:Bake the Whole Cake Teaching the alphabetic principle skill builds better readers • Teach letter-sounds (some think this is all phonics refers to) • Teach a decoding strategy: left to right, say it slowly sound by sound, say it fast, say the word. • Instruction on scaffolded recoding: sound out and read word • Instruction on fading the scaffold: teach to internalize the sounding out step and just read the word • Fluency building instruction: practice putting the whole piece together quickly and confidently. Salt Lake City, Utah

  10. Alphabetic Principle:Torching the cake Artificial ways to raise the NWF score without actually teaching the skill are like torching the cake – they don’t lead to better reading outcomes. • Memorizing and practicing the nonsense words on the next DIBELS assessment. Knowing tob dos et tuj kej does not help children learn to read better. • Telling children not to recode (read the words as a word) but instead just to say the letter sounds as fast as they can. (Recoding is the point of NWF) • Giving extra time or assistance. Salt Lake City, Utah

  11. Accuracy and Fluency with Connected Text:Torching the cake Artificial ways to raise the ORF score without actually teaching the skill are like torching the cake – they don’t lead to better reading outcomes. • Practicing the next DIBELS probe so they can read it fast does not help children learn to read better. • Encouraging children to read only the words they know: The and a an …. • Telling children to read as fast as they can. • Sending the passage home to practice. • Start reading the passage at the third sentence. • Giving extra time or assistance. Salt Lake City, Utah

  12. Bake the Whole Cake Squad Official Member of the DIBELS Bake the Whole Cake Squad I understand the importance of teaching the whole early literacy skill and not just the DIBELS measure. I will try to help others who are confused to focus instruction on the skills and not the probes to make a genuine difference in children’s reading outcomes. (Signature) Salt Lake City, Utah

  13. Role of Mid First Alphabetic Principle School A: • Odds of achieving ORF reading goal in March of First Grade when Established with NWF in December of First Grade are 11 out of 11, or 100%. • Odds of achieving ORF reading goal in March of First Grade when Deficit with NWF in December of First Grade are 0 out of 32, or 0%. Salt Lake City, Utah

  14. Similar Odds, Different Outcome School B: • Odds of achieving ORF reading goal in May of First Grade when Established with NWF in January of First Grade are 39 out of 43, or 90%. • Odds of achieving ORF reading goal in May of First Grade when Deficit with NWF in January of First Grade are 0 out of 4, or 0%. Salt Lake City, Utah

  15. Oregon Reading First 2004-2005: • Odds of achieving ORF reading goal in May of First Grade when low risk on NWF in the beginning of First Grade are 83%. • Odds of of achieving ORF reading goal in May of First Grade when at risk on NWF in the beginning of First Grade are 14%. Salt Lake City, Utah

  16. Overall Odds of Reading Outcomes r = .75 • Odds of being on track with ORF in end of first grade when Established with NWF in middle of first grade are 91% across all schools in DIBELS Data System 2001-2002. • Odds of being on track with ORF in end of first grade when Deficit with NWF in middle of first grade are 18% Salt Lake City, Utah

  17. Relation between percent at benchmark in winter of first grade (NWF Goal) and percent achieving 40 on ORF in spring of first grade (Fien, 2004). Each dot is a school. r = .85 Salt Lake City, Utah

  18. Four Purposes of Reading Assessments An effective, comprehensive, reading program includes reading assessments to accomplish four purposes: • Screening Measure: Brief assessment that focuses on critical reading skills strongly predictive of future reading growth and development, and conducted at the beginning of the school year with all children in grades K, 1, 2, and 3 to identify children likely to need extra or alternative forms of instruction. • Diagnostic Measure: Assessment conducted at any time during the school year only when more in-depth analysis of a student’s strengths and weaknesses is needed to guide instruction. Salt Lake City, Utah

  19. Four Kinds of Reading Assessments • Progress Monitoring Measure: Assessment conducted a minimum of three times a year or on a routine basis (i.e., weekly, monthly, or quarterly) using comparable and multiple test forms to (a) estimate rates of reading improvement, (b) identify children who are not demonstrating adequate progress and therefore require additional or different forms of instruction, and/or (c) compare the efficacy of different forms of instruction for struggling readers and thereby design more effective, individualized instructional programs for those at-risk learners. • Outcome Measure: Assessment for the purpose of classifying students in terms of whether they achieved grade-level performance or improved. Salt Lake City, Utah

  20. Efficient and Purposeful Assessment • Goal: Assessment maintained over time with the resources available, and in a way that is not so time-consuming that it compromises the instructional mission of the school. • One way to achieve efficient assessment is to have one measure accomplish screening, progress monitoring, and outcome purposes. • The initial screening can also serve as the first progress monitoring assessment. • Example: CTOPP, TOWRE, TPRI, CBM, and DIBELS (among others) have sufficient evidence for screening, progress monitoring, and outcome decisions in selected areas. Salt Lake City, Utah

  21. Outcomes Driven Model Provides a Decision Structure for Assessment Salt Lake City, Utah

  22. Using an Outcomes Driven Model to inform Instructional Decisions Outcomes Driven Model: Decision making steps 1. Identifying Need for Support 2. Validating Need for Instructional Support 3. Planning and Implementing Instructional Support 4. Evaluating and Modifying Instructional Support 5. Reviewing Outcomes for Individuals and Systems Salt Lake City, Utah

  23. 1. Identify Need for Support 2. Validate Need for Support 3. Plan and Implement Support 4. Evaluate and Modify Support 5. Review Outcomes Outcomes Driven Model in a Picture Outcomes Driven Model in a Picture Implement a Research-Based Intervention Increase intensity of Intervention: 1) Increase intervention fidelity 2) Increase time 3) Smaller Group Size Mid-year cutoff low risk Nonsense Word Fluency Mid-year cutoff at risk Salt Lake City, Utah

  24. 5. Reviewing Outcomes Key Decisions forOutcome Assessment: • Does each individual child have the early literacy skills predictive of successful reading outcomes? Is each individual child making adequate progress toward important literacy goals? • Does the school have a schoolwide system of instruction and support so their students achieve literacy outcomes? • Evaluate the schoolwide system (core curriculum and instruction,supplemental support, and intervention) for each step to identify strengths and targets of opportunity for improvement. Salt Lake City, Utah

  25. Amanda’s Progress In First Semester of First Grade Predicted End of First ORF: 9Actual End of First ORF: 8 • Amanda is making progress on alphabetic principle skills (phonics), but not enough progress to change her risk status. • Amanda is not making adequate progress. Sept Jan Salt Lake City, Utah

  26. Belinda’s Progress In First Semester of First Grade Predicted End of First ORF: 20Actual End of First ORF: 26 • Belinda is making progress on alphabetic principle skills (phonics), and reducing her risk level (moving from at risk to some risk). • Belinda is making adequate progress. Sept Jan Salt Lake City, Utah

  27. Calvin’s Progress In First Semester of First Grade Predicted End of First ORF: 32Actual End of First ORF: 38 • Calvin is making progress on alphabetic principle skills (phonics), and achieving the middle of first grade goal. (We want 3 consecutive points above the goal to be sure.) • Calvin is making exceptional progress. Sept Jan Salt Lake City, Utah

  28. Predictions for Amanda, Belinda, and Calvin • Predictions of first grade reading outcomes are based on 289 Oregon Reading First students whose progress was monitored at least 5 times in the first semester of first grade. • Predicted end of first grade reading outcomes are based only on NWF initial skills and slope of progress on NWF for the first semester of first grade. (Note, predictions are improved when use all DIBELS). • Initial skills on the alphabetic principle explained 24% of the variance in first grade reading outcomes. • Slope or rate of progress on alphabetic principle skills added an additional 35% of variance explained. Salt Lake City, Utah

  29. DIBELS for Schoolwide System Review:I am very anxious about using DIBELS for Accountability decisions • On the one hand, if our children are not making adequate progress in reading, I think we should stand up and take notice. • On the other hand, if students, teachers, schools, districts, or states are criticized or suffer consequences for DIBELS scores, they are less likely to use the information as designed to improve instruction. • DIBELS should never be used for student retention decisions. http://www.nasponline.org/information/pospaper_graderetent.html http://www.education.ucsb.edu/jimerson/retention/ • DIBELS should be used to identify targets of opportunity and provide support. Salt Lake City, Utah

  30. The purpose of DIBELS is to improve OutcomesEffect of Retention on Student Outcomes • No evidence of positive effects of retention on long-term achievement, self esteem, or adjustment • Negative effects of Retention: Retention increases the odds of • Increased odds of academic difficulty, especially in reading. • Increased odds of socio-emotional adjustment difficulties. • Increased likelihood of behavior problems especially in adolescence. • Increased risk of school drop out. Retention is one of the most powerful predictors of not completing high school. • Increased risk of being retained a second time. Students retained more than once have very low likelihood of completing high school. • Increased risk of “emotional distress, cigarette use, alcohol use, drug abuse, driving while drinking, use of alcohol during sexual activity, early onset of sexual activity, suicidal intentions, and violent behaviors” (NASP, 2003). • More likely to be unemployed, underemployed, or on public assistance. • More likely to be in prison. Salt Lake City, Utah

  31. DIBELS is very appropriate for Outcomes Assessment • Supports effective leadership to focus effort, set ambitious goals, provide support to achieve goals, emphasize outcomes, build teams. • School-based teams make decisions about how to improve the system of instruction. • Identify targets of opportunity – What grades? What tiers of the system? What core skill areas? • Focus on support – What can we provide to improve outcomes? Salt Lake City, Utah

  32. Focus on Support • What can we do as a system to support the effectiveness of the Schoolwide System of Instruction? • What would it take to help the school achieve literacy goals? • Professional development on essential components of early literacy? • More powerful interventions or supplemental materials? • Coaching to improve fidelity of implementation? • Additional resources to meet the needs of challenging students (e.g., ELL, high mobility)? • Administrative support to invest substantial time and resources to change outcomes? Salt Lake City, Utah

  33. SchoolwideSystem of Instruction and Support Core Curriculum and Instruction SupplementalSupport Intervention Salt Lake City, Utah

  34. Each Teacher Goals All Students Instruction Assessment Each Student All Teachers Salt Lake City, Utah

  35. Schoolwide System of Instruction and Support -- Core Curriculum and Instruction -- • Not just the reading curriculum selected but also the way it is delivered. • aka Primary Prevention or Benchmark Instruction • Primary Goal: Meet the needs of 80% of students in the school. If the schools has lots of children who need strategic or intensive support, the core curriculum and instruction will need to include many feature of strategic support and intensive intervention • Primary Step-Goal: Support all benchmark students to make adequate progress and achieve the benchmark goal. • Secondary Step-Goal: Support 50% of strategic students to achieve the benchmark goal. Salt Lake City, Utah

  36. Schoolwide System of Instruction and Support -- Supplemental Support -- • Additional time, smaller group, more intensive, supplemental or intervention program, delivered with fidelity. • aka Secondary Prevention or Strategic Support • Primary Goal: Meet the needs of 15% of students in the school who will need more support than the core curriculum and instruction can provide. • Primary Step-Goal: Adequate progress to reduce risk of reading difficulty. Support all strategic students to achieve the benchmark goal. Salt Lake City, Utah

  37. Schoolwide System of Instruction and Support -- Intervention -- • Additional time, smaller group, more intensive, supplemental or intervention program, delivered with fidelity. • aka Tertiary Prevention or Intensive Intervention • Primary Goal: Meet the needs of the 5% of students in the school who will need very intensive intervention to achieve literacy goals. • Primary Step-Goal: Accelerate learning and progress to support all intensive students to achieve the benchmark goal or reduce their risk of reading difficulty to strategic. If one step can get them to strategic, the next step can get them to benchmark. Salt Lake City, Utah

  38. Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Step 8 Step 3 Steps to Achieving Reading Outcomes Adapted from Good, R. H., Simmons, D. C., & Kame'enui, E. J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5, 257-288.

  39. Instructional Goals for Essential Components of Beginning Reading Essential DIBELS Goal Skill Timeline Component Indicator Level

  40. DIBELS Steps to Reading Success One Step per SemesterOne Goal per Step(Keep moving in the direction of the next goal) Step 8 - ORF 3 Step 7 - ORF 3 Step 6 - ORF 2 Step 5 - ORF 2 Step 4 - ORF 1 Step 3 - NWF 92 110 G3 ORF Step 2 - PSF 68 90 G2 ORF Step 1 - ISF 40 G1 ORF 50 NWF 35 PSF PSF 25 ISF Beg Mid End Beg Mid End Beg Mid End Beg Mid End Kindergarten First Grade Second Grade Third Grade (Howe, 2005) Salt Lake City, Utah

  41. 95% Schoolwide System Goal • A powerful schoolwide system goal is to have 95% of our students achieve the benchmark goals on time. • If 95% achieve the goal, the core curriculum and instruction, supplemental support, and intensive intervention are all doing their part. • We always are striving for 100%, but 95% means the system is working, the remaining students represent individual issues and challenges. • 45% achieving the goal means we need to revise our system. • One or two students not achieving the goal means we need to address their individual challenges. Salt Lake City, Utah

  42. So, How are we Doing in Teaching the Alphabetic Principle? National Target of Opportunity • A typical, median, school has 45% of first grade students achieving established Alphabetic Principle by the middle of first grade. Salt Lake City, Utah

  43. System-wide Comparisons of Middle of First Grade NWF Outcomes • How do our Alphabetic Principle outcomes compare to other schools around the country? 57% or more established on NWF in middle of first grade would be in the upper third. 34% or less established on NWF in the middle of first grade would be in the lower third. • Note: Based on N = 2316 schools with at least 40 students per grade level participating in the DIBELS Data System in 2003-2004. Salt Lake City, Utah

  44. System-wide Comparisons of End of First Grade ORF Outcomes • How do our Accuracy and Fluency with Connected Text outcomes compare to other schools around the country? 70% or more Low Risk on ORF and end of first would be in the upper third. 51% or less Low Risk on ORF at the end of first grade would be in the lower third. • Note: Based on N = 2298 schools with at least 40 students per grade level participating in the DIBELS Data System in 2003-2004. Salt Lake City, Utah

  45. If our school is not yet at a schoolwide system strength (95% at goal), Examine CSI Reports • Effectiveness Reports and CSI Reports examine each Tier of our 3 Tier system of support. • Tier 1 – Core curriculum and instruction examines outcomes for our students with a Benchmark instructional recommendation. • Tier 2 – Supplemental Support examines outcomes for our students with a Strategic instructional recommendation. • Tier 3 – Intensive Intervention examines outcomes for our students with an Intensive Support instructional recommendation. • The purpose reviewing each tier of our system is to identify a target of opportunity where we can strengthen a level of our system to improve outcomes overall. Salt Lake City, Utah

  46. DIBELS Summary of Effectiveness Reports4 Ways to Achieve Adequate Progress Time 1 (e.g., Fall) Time 2 (e.g., Winter) At Risk Effective Intensive Intervention Intensive(Tier 3) 1. Some Risk 2. Low Risk At Risk Effective Core and Supplemental Support Some Risk Strategic(Tier 2) 3. Low Risk At Risk Effective Core Curriculum and Instruction Some Risk Benchmark(Tier 1) 4. Low Risk (Baker, 2005) Salt Lake City, Utah

  47. CSI Report – Identify Targets of Opportunity • Core Curriculum and Instruction – Benchmark Students • Strength – 95% of Benchmark Students Achieve Goal • Relative Strength – Upper Third • Needs Support – Middle Third • Needs Substantial Support – Lower Third • Supplemental Instruction – Strategic Support Students • Strength – 80% of Strategic Students Achieve Goal • Relative Strength – Upper Third • Needs Support – Middle Third • Needs Substantial Support – Lower Third • Intensive Intervention – Intensive Support Students • Strength – 80% of Intensive Students are Emerging or Achieve Goal • Relative Strength – Upper Third • Needs Support – Middle Third • Needs Substantial Support – Lower Third Salt Lake City, Utah

  48. Core Instruction Should Keep Students On Track for Literacy Outcomes • Students that are on track (a Benchmark Instructional Recommendation) should achieve the next literacy goals. • Strength of the core curriculum and instruction is the backbone of the schoolwide instructional system. • Adequate progress of Strategic Students and Benchmark Students signal a strong system of core curriculum and instruction. • Many benchmark students not making adequate progress signals a target of opportunity. • How much progress can we reasonably expect? Salt Lake City, Utah

  49. Summary of Effectiveness Report • Available from http://dibels.uoregon.edu • Summary of Effectiveness Report is available through the DIBELS Data System. • For Wireless Generation Palm users, there is a data bridge between the mClass data system and the DIBELS Data System. Palm users can access all of the reports from the DIBELS Data System. • Or, examine the model and count the number of Benchmark Students at the beginning of each step who achieve each goal for that step, and divide by the number of Benchmark Students at the beginning of the step. Salt Lake City, Utah

More Related