Progress Monitoring

2. Progress Monitoring Strategies for Writing Individual Goals in General Curriculum and More Frequent Formative Evaluation Mark Shinn, Ph.D.Lisa A. Langell, M.A., S.Psy.S.

3. Big Ideas About Frequent Formative Evaluation Using General Outcome Measures and the Progress Monitoring Program One of the most powerful interventions that schools can use is systematic and frequent formative evaluation. Benchmark Assessment is not enough for some students because they may be in ineffective programs too long. (3 mos +) The solution is to write individualized goals and determine a feasible progress monitoring schedule. The core of frequent progress monitoring is: Survey-Level Assessment Goal setting using logical educational practices Analysis of student need and resources for determining progress monitoring frequency.

4. Formative Assessment • Formative Assessment: Process of assessing student achievement during instruction to determine whether an instructional program is effective for individual students. • When students are progressing, keep using your instructional programs. • When tests show that students are not progressing, you can change your instructional programs in meaningful ways. • Has been linked to important gains in student achievement (L. Fuchs, 1986) with effect sizes of .7 and greater.

5. Systematic formative evaluation requires the use of: • Standard assessment tools… • That are the same difficulty • That are Given the same way each time.

6. More Severe Achievement Problems and/or More Resource Intensive Programs Require More Frequent Formative Evaluation Benchmark Testing (3 - 4 x Per Year) is not enough for some students.

7. With Very Low Performers, Not Satisfactory to Wait This Long!

8. Programs That are More Resource Intensive… • Title I, English Language Learning, Special Education • Should monitor student outcomes more frequently than the Benchmark Testing schedule.

9. Formative Evaluation of Vital Signs Requires Quality Tools Technical adequacy (reliability and validity); Capacity to model growth (able to represent student achievement growth within and across academic years); Treatment sensitivity (scores should change when students are learning); Independence from specific instructional techniques (instructionally eclectic so the system can be used with any type of instruction or curriculum); Capacity to inform teaching (should provide information to help teachers improve instruction); Feasibility (must be doable).

10. Thinking About A Student’s Data Sample Student: Melissa Smart 3rd grade student Progress Monitor

11. 8 Melissa Smart 110 92 77 50 34

12. Formative Evaluation—Is simply data enough?

13. Formative Evaluation: Is data and a goal enough?

14. Formative Evaluation: Are data, goals & trends enough?

15. Formative Evaluation is Impossible without all data:Goals Make Progress Decisions Easier

16. Current Goal Setting Practices Are Unsatisfying! • Do you like these IEPs? • I do not like these IEPs • I do not like them Jeeze Louise • We test, we check • We plan, we meet • But nothing ever seems complete. • Would you, could you • Like the form? • I do not like the form I see • Not page 1, not 2, not 3 • Another change • A brand new box • I think we all • Have lost our rocks!

17. Need Shift to Few But Important Goals • Often Ineffective Goal Smorgasboard! • Student will perform spelling skills at a high 3rd grade level. • Student will alphabetize words by the second letter with 80% accuracy. • Student will read words from the Dolch Word List with 80% accuracy. • Student will master basic multiplication facts with 80% accuracy. • Student will increase reading skills by progressing through Scribner with 90% accuracy as determined by teacher-made fluency and comprehension probes by October 2006. • To increase reading ability by 6 months to 1 year as measured by the Woodcock Johnson. • Student will make one year's growth in reading by October 2006 as measured by the Brigance. • Student will be a better reader. • Student will read aloud with 80% accuracy and 80% comprehension. • Student will make one year's gain in general reading from K-3. • Students will read 1 story per week.

18. Improving the Process of Setting Goals for Formative Evaluation Set a few, but important goals. Ensure goals are measurable and linked to validated formative evaluation practices. Base goal setting on logical educational practices.

19. Reduce the Number of Goals to a Few Critical Indicators Reading In (#) weeks (Student name) will read (#) Words Correctly in 1 minute from randomly selected Grade (#) passages. Spelling In (#) weeks (Student name) will write (#) Correct Letter Sequences and (#) Correct Words in 2 minutes from randomly selected Grade (#) spelling lists. Math Computation In (#) weeks (Student name) will write (#) Correct Digits in 2 minutes from randomly selected Grade (#) math problems. Written Expression In (#) weeks (Student name) will write (#) Total Words and (#) Correct Writing Sequences when presented with randomly selected Grade (#) story starters.

20. Ensure the Goals are Measurable and Linked to Validated Formative Evaluation Practices Goals should be based on quality tests like CBM. Based on validated practices such as how often, how many samples, etc.

21. Conducting a Survey Level Assessment Students are tested in successive levels of general curriculum, beginning with their current expected grade placement, until a level at which they are successful is determined.

22. John5th grader:5th grade passage 26/12 John3rd grade passage62/4 John4th grade passage49/7 Conducting a Survey Level Assessment

23. Base Goal Setting on Logical Educational Practices Example of PLEP statement: John currently reads about 26 words correctly from Grade 5 Standard Reading Assessment Passages. He reads Grade 3 reading passages successfully; 62 correct words per minute with 4 errors, which is how well beginning 3rd grade students read this material.

24. Goal Setting Strategies Current Performance Information based on Survey-Level Assessment (SLA). Know the Time Frame for the Goal. Determine a Future Performance Level.

25. Setting the Time Frame, Goal Level Material, and Criterion • Time Frame • End of Year (At Risk or Grade-Level Expectations) • In 18 Weeks… • Annual IEP Goals (Special Education) • In 1 year… (or) In 32 Weeks…

26. Setting the Goal Material • Logical Task-- • Matching or Not Matching Expected Grade Placement • Title I: Fourth Grader--Grade 4 Material? • Grade 4: Special Education Student--Grade 4 Material?

27. When Grade-Level Expectations Are Not Appropriate • Consider the Severity of the Discrepancy • Consider the Intensity of the Program

28. Determining the Criterion for Success: Options to use • Local Benchmark Standards. • Linkage to High Stakes Tests. • Normative Growth Rates. • Developing Your Own Sample of Standards.

29. 1. Benchmark Standards Advantages and Disadvantages

30. 2. Linkage to High Stakes Standards Advantages and Disadvantages

31. Normative Growth Rates Criterion for Success = Score on SLA + (Grade Growth Rate times # of Weeks) Score on SLA (30)+ (Ambitious Grade Growth Rate (2.0) times # of Weeks (32) Or 30 + (2.0 * 32) or 30 + 64 = Annual goal of 94 WRC

32. 3. Growth Rate Standards Advantages and Disadvantages

33. Developing Your Own Sample of Standards Developing a Small Local Norm

34. Benchmark Standards Advantages and Disadvantages

35. How Frequently to Assess? Balancing IDEAL with FEASIBLE

36. Making Data-Based Decisions With Progress Monitor • Need at LEAST 4-7 data points before making programming decision—…and you may sometimes want to collect more if you are uncertain. • Err on the side of caution • Criteria To Consider: • Trendline meets Aimline for ultimate goal:Consider return to LRE. • Trendline and AIMline will intersect in relatively near future? • Keep with current intervention until goal is reached. • Trendline exceeds AIMline? • Consider increasing goal or difficulty level. • Trendline not going to intersect AIMline—moves in opposite direction: Consider adding additional intervention, changing variable, and/or insensifying program changes (LRE).