Using Data to Improve Student Achievement

1. Using Data to Improve Student Achievement Summer 2006 Preschool CSDC

2. Outcomes • Know why we need to look at data • Identify two types of tests • Understand three types of scores • Understand Summative & Formative Assessments • Be able to interpret Summative Assessment Reports • Know how to use data in instructional planning for increased student learning

3. Why Look at Data? The purpose of data is to give educators INSIGHT!

4. Types of Tests • Norm-Referenced Test (NRT) • Criterion-Referenced Test (CRT)

5. What is aNorm-Referenced Test (NRT)? • A standardized assessment in which all students perform under the same conditions. • It compares the performance of a student or group of students to a national sample of students at the same grade and age, called the norm group.

6. What is aCriterion-Referenced Test (CRT)? • An assessment comparing one student's performance to a specific learning objective or performance standardand not to the performance of other students. • It tells us how well students are performing on specific goals or content standards rather than how their performance compares to a national or local norming group.

7. Summary NRT and CRT

8. Types of Scores Raw Scores Scale Scores Gain Scores

9. Raw Score (RS) • The number of items a student answers correctly on a test. • John took a 20 item mathematics test (where each item was worth one point) and correctly answered 17 items. • His raw score for this assessment is 17.

10. Scale Score (SS) • Mathematically converted raw scores based on level of difficulty per question. • For FCAT-SSS, acomputer program is used to analyze student responses and to compute the scale score. • Scale Scores reflect a more accurate picture of the student’s achievement level.

11. Commonly referred to as “Learning Gains” The amount of progress a student makes in one school year. Gain Scores

12. Learning Gains: Who Qualifies? • All students with a pre- and post-test, including all subgroups (ESE, LEP, etc.). • All students with matched, consecutive year (i.e. 2005 & 2006) FCAT SSS results, grades 4-10, who were enrolled in the same school surveys 2 & 3 (FTE).

13. Learning Gains: Which Scores? • Gains apply in reading and math, not writing or science. • Pre-test may be from same school, same district, or anywhere in the state.

14. Learning Gains: What equals Adequate Yearly Progress (AYP)? A. Improve FCAT Achievement Levels from 2005 to 2006 (e.g. 1-2, 2-3, 3-4, 4-5) OR B. Maintain “satisfactory” Achievement Levels from 2005-2006 (e.g. 3-3, 4-4, 5-5) OR C. Demonstrate more than one year’s growth within Level 1 or Level 2 - determined by DSS Cut Points (not applicable for retained students)

15. Developmental Scale Score Gains Table (DSS Cut Points)

16. Learning Gains: Retainees A retained student can only be counted as making adequate progress if he/she: Moves up one level. (e.g. 1-2, 2-3, 3-4, 4-5) Maintains a level 3, 4, or 5. REASON “A” REASON “B”

17. Learning Gains: Activity Using the data on the following table, determine: • which students made a learning gain • what percentage of the teacher’s students made a learning gain

18. Data Display for FCAT Reading Results

19. Teacher Learning Gains Based on Data Display • 5 out of 7 students made learning gains. • 71% of this teacher’s students made learning gains and add points towards the school’s grade. • No points are given to the school for Student F because he was retained and stayed within level 1 – even though he made significant gains in DSS points. • No points are given to Student G because he decreased a level.

20. Class Record Sheet for Learning Gains

21. Results (Summative) Data used to make decisions about student achievement at the end of a period of instruction. Process (Formative) Data gathered at regular intervals during the instructional period; used to provide feedback about student progress and to provide direction for instructional interventions. Types of Data

22. A Closer Look at Results Data Examples: FCAT SAT 10

23. FCAT Parent Report

24. Formative Data Sources Quizzes Chapter Tests FORF/Fluency Reading Assessments

25. What tools do we have? • FCAT Inquiry (Summative) • Teacher Tools for Data Collection (Can be Summative or Formative) • Histogram • Pareto Chart • Run Chart • Scatter Diagram • Item Analysis

26. Histogram • Bar chart representing a frequency distribution of student scores • Heights of the bars represent number of students scoring at same level/score • Used to Monitor progress

27. Histogram: Midterm in 8th grade Reading 16 14 12 10 Frequency 8 6 4 2 0 >50% 50-59% 60-69% 70-79% 80-89% 91-100% Grade

28. Histogram: Grade Distribution in 8th Grade English 70 60 50 40 Frequency 30 20 10 0 0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100 Grade

29. Run Chart Use to: Monitor progress over time Display data in simplest form

30. Run Chart: Number of Words Spelled Correctly on Weekly Quiz 180 175 170 165 Number of words 160 155 150 145 140 1 2 3 4 5 6 7 8 9 Week

31. Class Run Chart: Percent of Students Averaging at Least 80% on Weekly Math Quizzes 100 90 80 70 60 Percent w/ avg. of at least 80% 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 Week Class Goal: By the end of 9 weeks, 100% of our class will have an average of at least 80% on our weekly math quizzes.

32. Scatter Diagram: Quiz Average vs. Test Average 100 90 80 Quiz Average 70 60 50 50 60 70 80 90 100 Test Average Scatter Diagram

33. Scatter Diagram: Number of Siblings vs Grade Point Average 4.50 4.00 3.50 3.00 2.50 Grade Point Average 2.00 1.50 1.00 0.50 0.00 0 1 2 3 4 5 6 Number of Siblings

34. Scatter Diagram: Hours of Sleep vs Mistakes on Test 12 10 8 Mistakes 6 4 2 0 0 1 2 3 4 5 6 7 8 9 10 Hours of Sleep

35. Item Analysis Use to: Determine mastered content Determine most common mistakes

36. CLASSROOM TEST ANALYSIS

37. ITEM ANALYSIS ACTIVITY

38. Pareto Chart Use to: Rank issues in order of occurrence Decide which problems need to be addressed first Find the issues that have the greatest impact Monitor impact of changes

39. Pareto Chart: Types of mistakes in Division Problems 100 100 90 90 80 80 70 70 60 60 Cumulative percentage Percent 50 50 40 40 30 30 20 20 10 10 0 0 Incorrect multiplication Incorrect subtraction No decimal Other Mistake

40. Pareto Chart Mistakes by Subtest 120% 100% 80% Cumulative Percentage Percentage of Mistakes 60% 40% 20% 0% Reference and Research Author's Purpose Compare / Contrast Cause and Effect Main Idea Subtest

41. Data analysis provides: Insight and Questions ? ? ? ? ?

42. Questions to Ponder… • What question are we trying to answer? • What can we tell from the data? • What can we NOT tell from the data? What else might we want to know? • What good news is here for us to celebrate? • What opportunities for improvement are suggested by the data? Adapted from Getting Excited About Data, Edie Holcomb www.corwinpress.com

43. Action Thinking Maps Learning Modalities Provides Marzano Strategies Peer Tutor Answers! CRISS Strategies Cloze activities Cooperative Learning

44. Do It Again! Steps to Improvement ACT Make improvements. STUDY Analyze the results. DO Implement the plan. PLAN What information have I gained from my data? What interventions can I put In place?

45. Do It A S D Again! P Personal Action Plan What data can I access? What tools can I use to help me monitor progress toward our class goals? What/who else do I need to help me? What is my start date? How will I evaluate the results?