Welcome

1. Welcome Data Driven Decisions Moving from 3D to D3

2. Data Driven Decisions Moving from 3D to D3 Malcolm Thomas Director, Evaluation Services Escambia School District

3. Presentation Agenda 3D Data Driven Decisions - Review Power of Data Analysis Moving to D3 - Formative Assessments Review of 3D Resources

4. Contact Information ... Malcolm Thomas mthomas@escambia.k12.fl.us (850) 469-5386 Resources Link ... www.escambia.k12.fl.us/adminoff/eval

5. If a pebble is dropped into a pond . . . . • How much water is impacted?

6. “Insanity is doing the same thing over and over expecting a different result.”Albert Einstein

7. Digging In the Data One of the first steps: Develop a common vocabulary that will lead to common understanding.

8. Scale Score FCAT NRT NPR CRT Percentile Stanine Raw Score DVSS NCE

9. Florida Students are administered two(2) tests for Reading and Mathematics • Norm Referenced Test (NRT) • Criterion Referenced Test (SSS) • Administered to all students in Grades 3 - 10

10. Scale Scores Use to compare year to year and to interpret trends in performance National Percentile Rank (NPR) % in national sample scoring at or above student (ranks) NOT equal interval Normal Curve Equivalent (NCE) Equal interval scale from 1 to 99 with average of 50 IS equal interval Stanine Scores divided into 9 equal groups Scores of 4 – 6 are considered average NRT Score Types

11. Normal Distribution Curve“A Band of Scores” 68% 95% 99% -2 -1 1 0 -3 2 3 MEAN  = Standard Deviation

12. Criterion Referenced Test (SSS) • what test takers can do and what to others • report how well students are doing relative to a pre-determined performance level on a specified set of educational goals or outcomes

13. FCAT CRT Scores (SSS)Scale Scores 100 - 500

14. FCAT SSS Scale Scores • Scale scores have no meaning in isolation • Scale Scores are used to determine achievement levels • FCAT CRT (SSS) Scale score range is 100 - 500

15. FCAT Developmental Scale Scores (DVSS) • Created to help understand students' year-to-year progress. • DVSS range from 86 to 3008. • Students should receive higher scores as they move from grade-to-grade according to their increased achievement. • Use DVSS to monitor academic progress each year.

16. DVSS Linear Progression

17. Let’s Review Scores ByTypes by Test NRT • Raw Score • Scale Score • NPR • NCE • Stanine CRT (SSS) • Raw Score • Scale Score • DVSS • Achievement Level

18. Learning Gains • Measure individual student progress from year to year • Uses the Developmental Scale Score and Achievement Level • Compares current scores to scores of previous year

19. Making a Learning Gain… • Improve FCAT at least one achievement level (example move from Level 1 to Level 2 or Level 3 to Level 5) • Maintain satisfactory achievement levels (Levels 3, 4, 5) • Demonstrate more than one year’s growth within Level 1 or Level 2 (using DVSS Gains Table) Retained students MUST increase 1 achievement level or maintain satisfactory achievement level (cannot use DVSS method for retained students)

20. Developmental Scale Gain Developmental Scale Scores must be at least one point greater than the above cut-off scores for a student to make a Learning Gain. This method of Learning Gain calculation only applies to promoted students maintaining achievement 1 or 2.

21. Monitoring Student Progress … • Developmental Scale Scores (DVSS) for consecutive Years • NRT NCE scores for consecutive years • Learning Gains for consecutive years

22. Content Clusters The way the FCAT SSS breaks down the subject area tests. The number of items varies across disciplines and grade levels.

23. SSS Scores are used in School Grade Calculations Changes are coming to School Grade calculations in 2007

24. School grade will consist of a total of eight components in 2007 • Four (4) Proficiency components: • Reading • Math • Writing • Science **NEW** • Four (4) Learning Gain Components: • Reading Learning Gain • Lowest 25% Reading Learning Gain • Math Learning Gain • Lowest 25% Math Learning Gain **NEW**

25. Revised School Grading Scale

27. THE POWER OF DATA ANALYSIS Assessthe current and future needs of students Decidewhat to change Determineif goals are being met Engage in continuousschool improvement Identifyroot causes of problems Promote accountability

28. Leadership is the key … Source: Brandeis University

29. BARRIERS TO DATA ANALYSIS • Lack of training in data use • No uniform data collection • Lack of leadership at the school and district level • Outdated technology • Unclear priorities • Distrust of data use

30. Elements for Effective Use of Data • Create collaborative / trusting relationships • Engage in data driven dialogue and collaborative inquiry • Learn what you can from standardized tests • Use multiple measures • Examine data in variety of ways

31. Collaborative Inquiry

32. Questions usually lead to …. . . . more questions and more data collection.

33. When analyzing dataA Picture is Worth …

35. Collect Multiples Measures • Example: • Grade 3 Reading FCAT Scale Scores • Grade 3 Reading National Percentile Ranks • Or Grade 3 Reading FCAT Achievement Level Percentages for multiple years

36. Multiple Years of Data

37. Comparing FCAT SSS Reading to Reading NRT

38. Other Examples of Multiple MeasuresCompare Student Reading or Math Scores with: • Averages of State, District or School • Attendance Data • Achievement Scores And Discipline Referrals • GPA • Content Areas By Grade Level

39. FCAT Myths to Avoid • Performance for individual student assessments should remain the same or improve • FCAT Proficiency equals grade level • FCAT Proficiency levels are the same for each grade level • All data are accurate

40. Moving ... From 3D . . . Data Driven Decisions to . . . D3 Data Driven Decisions

41. Summative Assessments • Summative assessments attempt to summarize student learning at a specific point in time • Most standardized tests are summative • Excellent tool for shaping district / school or course goals • Major disadvantages • time lag between assessment and results • lack of specific information that can be used by teachers for instruction

42. Formative Assessments • Occurs when teachers provide information to students in ways that enable the student to learn better, or when students can engage in a similar, self- reflective process • Primary purpose support high quality learning • Most effective for lower performing students; although all students benefit

43. Formative Assessments • Major Advantages • Can occur in real time • Maximize teachable moments in the classroom • Both teachers and students receive information necessary for improvement

44. Profound Mistake • Students NOT included as active participants in the assessment process in order for achievement to improve

45. Remedies Create a balanced system of summative assessments of learning and formative assessments for learning

46. A Balanced Approach Assessment of Learning How much have students learned as of a particular point in time? Assessment for Learning How can we use assessment to help students learn more?

47. High Quality Classroom Assessment Teachers should be asking: • Why am I assessing? • What am I assessing? • What is the best assessment method? • How do I communicate the assessment results?

48. Paper-pencil tests Exhibits Interviews Surveys Observations Other measures Assessments Tools, strategies, and techniques to analyze each student’s demonstration of accomplishments of specific goals and criteria. Would you rather be at the beach? Good assessment requires variety to ensure opportunities for success for all students.

49. Traditional Assessment Methods • Selected response • Extended written response • Performance assessment • Personal oral communication

50. Selected Response Students select an answer from a list or generate a very brief answer. Examples: Multiple Choice True / False Matching Fill in the blank Label a diagram