Data Warehousing How do we use the data now that we have it?

1. Data WarehousingHow do we use the data now that we have it? David Rutherford Director, Instructional Support Services ONC BOCES May 18, 2007

2. Where Are We Going? • Data Analysis in the ONC BOCES Region • Using the 3-8 Assessments for School Improvement • The Two Holy Grails of the 3-8 Assessments

3. Data Analysis at ONC BOCES • Our RIC – South Central RIC, Binghamton, NY • Data Analysis Service – Dr. Carol Dean • Monthly Meetings – Carol, Instructional Support, Staff Developers

4. School Improvement and Assessments "not everything that counts can be counted-- not everything that can be counted counts“ - Albert Einstein

5. School Improvement and Assessments Over-reliance on the 3-8 Assessments

6. School Improvement and Assessments Limited Nature of the 3-8 Assessments • Short Assessment – not everything is assessed • Turn-around time – cannot be used effectively as a formative assessment

7. School Improvement and Assessments

8. School Improvement and Assessments

9. The Two Holy Grails of Assessment Holy Grail Number 1 The State Assessments are verifications of existing assessments within the school/district

10. The Two Holy Grails of Assessment Holy Grail Number 1 Professional Learning Communities

11. The Two Holy Grails of Assessment Professional Learning CommunitiesRegional ONC BOCES Plan • Communal Organization of Teachers • Common Understanding of Standards at Grade Level • Development of Common Formative Assessments • Collective Evaluation of Results • Change in Classroom Practice

12. The Two Holy Grails of Assessment Coleman et al. (1966) “Equality of Educational Opportunity in the United States.” • Studied 600,000 children at 4,000 schools • An act of Congress during the civil rights Era. • Had the resources, authority and visibility of the federal government behind the study • The finding, neatly summarized, meshed with what many people saw with their own observations • The credibility, congruence and capacity of the research gave it immense staying power – to this day with the general public

13. The Two Holy Grails of Assessment Coleman – cont. “[S]chools bring little influence to bear upon a child’s achievement that is independent of his background and general social context” (Coleman et al., 1966, p.325).

14. The Two Holy Grails of Assessment Holy Grail Number 2 The State Assessments can be used to show growth, giving a better picture of school quality and over time-improvement

15. The Two Holy Grails of Assessment Holy Grail Number 2 – It’s Coming • NCLB Reauthorization • Article VII (Contract For Excellence)

16. The Two Holy Grails of Assessment Holy Grail Number 2 – Achievement vs. Progress • Achievement model: Determine whether an increasing percentage of students are gaining proficiency over time. • Growth model: Determine how much progress groups of students are making over time.

17. The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Measures a student’s performance at a single point in time Progress – Measures a student’s progress between two points in time

18. The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Relates to a student’s family background Progress – Not related to a student’s family background

19. The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Compares students’ performance to a standard Progress – Compares students’ performance to their own prior performance

20. The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Critical to a student’s post secondary opportunities Progress – Critical to ensuring a student’s future academic success

21. The Two Holy Grails of Assessment Holy Grail Number 2 – Achievement vs. Progress • Achievement model: Determine whether an increasing percentage of students are gaining proficiency over time. • Growth model: Determine how much progress groups of students are making over time.

22. The Two Holy Grails of Assessment Holy Grail Number 2 – Growth and the 3-8 Assessments Can we show growth now that we have longitudinal data? Problematic – why?

23. The Two Holy Grails of Assessment Holy Grail Number 2 – Vertically Moderated vs. Vertically Scaled Vertically Moderated – Meeting the “cut point” at one grade level indicates that the student is prepared for instruction at the next grade level. ** Student progress is measured from grade-to-grade relative to proficiency in meeting the standards

24. The Two Holy Grails of Assessment Holy Grail Number 2 – Vertically Moderated vs. Vertically Scaled Vertically Scaled – Meeting the same “cut point” in two years is an indication of one full year of growth

25. The Two Holy Grails of Assessment Holy Grail Number 2 – Vertically Moderated vs. Vertically Scaled NYS Assessments are Vertically Moderated Growth Model requires vertical scaling

26. The Two Holy Grails of Assessment Holy Grail Number 2 – Growth vs. Value Added • Growth models generally refer to models of school accountability that assess the progress of a cohort of students over time with the intent of measuring the progress these students have made (This year’s fourth graders compared to their performance last year as third graders.) • Value added models generally refer to a specific type of growth model in which student demographic data or other statistical controls are used to attempt to analyze the specific effects of a particular school, program, or teacher on student learning

27. The Two Holy Grails of Assessment Holy Grail Number 2 – Achievement vs. Progress Recap • Achievement - Where students need to be at the end of P-12 • Growth - How they get there

28. The Two Holy Grails of Assessment Value Added – SAS and Battelle for Kids • SAS – Dr. Bill Sanders developed the Value-Added Model • Battelle for Kids – Non-profit responsible for VA Professional Development • Capital Region and ONC BOCES – Grant to Pilot VA in New York State

29. School Improvement and Assessments More Information http://www.edtrust.org http://www.battelleforkids.org

30. School Improvement and Assessments "not everything that counts can be counted-- not everything that can be counted counts“ - Albert Einstein