The Oregon DATA Project

1. The Oregon DATA Project Direct Access to Achievement Evaluation Plan

2. Initiatives • The Teaching Learning Connection • KIDS and Regional Data Warehouses • Longitudinal Growth Project • DATA Project • Common goal: STUDENT ACHIEVEMENT

3. Collaboration The Teaching Learning Connection Establishes school-based processes and infrastructure to use data The KIDS Project Develops a data structure to streamline data acquisition and reporting • The Growth Project • Builds capacity to use and apply longitudinal data • Supporting Progress Toward the Oregon Diploma The Oregon DATA Project Builds capacity to apply the correct strategies, and to use the correct data at the correct time

4. DATA ProjectStakeholder Input • Carmichael Consulting • 3-month state-wide field study, 2008 • Nearly 200 stakeholders • Essential Question: • “What do we need to improve the use of data to advance student achievement in Oregon?”

5. DATA ProjectStakeholder Input: 5 macro-themes emerged • Statewide with minimum training • Training delivery • A central repository of student data • Common technical policies/tools • State policies and leadership

6. DATA ProjectResponse to identified needs • Gap analysis -- Creation of Professional Development Road Map • Strand 1: Creating a Data Culture • Administrators, district leaders, teacher leaders • Strand 2: Using Data to improve learning in districts and schools • Administrators, district leaders, teacher leaders • Strand 3: Using Data to improve learning in the classroom. • Principals, classroom teachers

7. DATA ProjectDeveloping Strand 1: Guiding Questions • What are the essential data needed to ensure continuous improvement? • What are the sources of these data in your educational setting? • What are the most effective and efficient ways to organize data to make decisions that realize ambitious goals? • How do you represent data in a meaningful way when interpreting and sharing with others? • How can longitudinal data be used to investigate student and systemic changes over time? • How can data be used appropriately to predict student performance? • How can data be used to evaluate program and material effectiveness? • How do you build and support district- and school-level data teams?

8. DATA ProjectDeveloping Strand 2:Guiding Questions • How is your school or district doing as a learning institution? • What evidence do you have that all students are learning? • Do you know why you are getting the results you currently have? • In what ways do administrators and teachers decide what data to collect and how to use data to make decisions? • How can growth data be used to design meaningful CIP and SIP plans? • How are data represented and communicated to different stakeholders?

9. DATA ProjectDeveloping Strand 3: Guiding Questions • What are the characteristics of quality interim and formative assessment? • How do you select assessments that align with your curriculum and instructional goals? • What are the appropriate uses of summative and formative assessment? What the common misuses of summative and formative assessment? • What comparisons are appropriate between classroom assessments and standardized test data? What comparisons are inappropriate? • How can data be used suitably to predict student performance? • How do you use results obtained from monitoring student progress over time to make instructional adjustments and design effective instructional interventions? • How are data represented and communicated to different stakeholders to enhance collaborative efforts to improve student achievement?

10. DATA ProjectGrounding Strands 1 - 3 • Ground the work to the development of Continuous Improvement Plans (CIP) and School Improvement Plans (SIP) • CIP • Communicates to all stakeholders the process of measuring and achieving improvements in a school district’s performance. • SIP • Communicates to all stakeholders the processes and strategies to be implemented at the classroom and school level to meet the goals of the district’s CIP.

11. DATA ProjectOur Theory of Change • Using appropriate data can influence instruction and improve student achievement • Using appropriate data can lead to improved and focused CIP and SIP plans. • CIP and SIP can guide and focus the work of districts, schools, and classrooms toward improved student achievement. • How will we know that it is happening? • Evaluation that yields measurable effects.

12. DATA ProjectEvaluation Plan • Council of Chief State School Officers (CCSSO) analysis of professional development programs (2008) • Quality of implementation • Gain in content knowledge and pedagogy skills • Change in instructional practice • Improvement in student achievement • Significant effects in programs with content focused professional development, sufficient time, and in school component (supports) • Important to plan purposeful evaluation

13. DATA ProjectEvaluation Plan • Quality of implementation • Outcome Measure • Change in level-of-use • Study Design • Base-line level-of-use data at the beginning • Follow up assessments at half-year, full-year, and two full-years later • Desired Finding • Has there been an increase in the use of strategies

14. DATA ProjectEvaluation Plan • Quality of implementation • Gain in content knowledge • Outcome Measure • Gain in knowledge and process • Study Design • Session evaluations will gather reactions and gain • Desired Finding • Session feedback will provide data that can be used to strengthen the training

15. DATA ProjectEvaluation Plan • Quality of implementation • Gain in content knowledge • Change in practice • Outcome Measure • Changes in CIP and SIP quality • Study Design • Sampling of plans assessed against a rubric combined with data gathered on level-of-use • Desired Finding • Increased level-of-use results in improved quality of CIP and SIP documents

16. DATA ProjectEvaluation Plan • Quality of implementation • Gain in content knowledge • Change in practice • Improvement in student achievement • Outcome Measure • OAKS results at building and district levels • Study Design • Sampling of districts to measure CIP/SIP quality, level-of-use, and OAKS results. Multiple year view. • Desired Finding • Quantify the connection between level-of-use, CIP/SIP quality and student achievement.

17. DATA ProjectEvaluation Plan • Quality of implementation • Gain in content knowledge • Change in practice • Improvement in student achievement • Content focused professional development • Outcome Measure • Providing adequate time in training to implement strategies • Study Design • Half-year follow up assessment for additional support • Desired Finding • Districts/schools needing additional support are identified

18. DATA ProjectEvaluation Plan • Quality of implementation • Gain in content knowledge • Change in practice • Improvement in student achievement • Content focused professional development • Plan purposeful evaluations • Outcome Measure • This evaluation plan • Study Design • Input and reactions from the field • Desired Finding • Providing measureable results that connect to student achievement

19. DATA ProjectEvaluation Plan: Key Components • Participant Information • Name • Email • School • District Name • ESD Name • Their Role(s) • Teacher, • Building Administrator, • Central Office Administrator, • Assessment Coordinator, • Student Information System Coordinator, • SIP Team Member, • CIP Team Member, or • Data Entry Personnel

20. DATA ProjectAnalysis Plan: Common Program Awareness

21. DATA ProjectEvaluation Plan: Key Components • Session Evaluation • Gusky recommends assess in two categories: • Participant’s Reactions • Content • Process • Context • Participant’s learning • Cognitive Goals (knowledge and understanding) • Psychomotor (skills and behaviors) • Affective (attitudes and beliefs) • Assist in session improvement and validate instruction.

22. DATA ProjectEvaluation Plan: Key Components • Pre-Assessment • Level of Use data (Gusky model) • Use the guiding questions as Focus Areas for statements • Quantify for analysis purposes

24. DATA ProjectGuiding Questions Strand 1 • What is the connection between data analysis, the creation of CIP and the creation of SIP? • What are the essential data needed to ensure continuous improvement? • What are the sources of these data in your educational setting? • What are the most effective and efficient ways to organize data to make decisions that realize ambitious goals? • How do you represent data in a meaningful way when interpreting and sharing with others? • How can longitudinal data be used to investigate student and systemic changes over time? • How can data be used appropriately to predict student performance? (held for strand 3) • How can data be used to evaluate program and material effectiveness? • How do you build and support district- and school-level data teams?

25. DATA ProjectGuiding Questions Strand 1 example • What is the connection between data analysis, the creation of CIP and the creation of SIP? • Each year our district completes a full analysis of OSAT results as well as local student achievement data in preparation for reviewing and writing the Continuous Improvement Plan. • Before developing a School Improvement Plan, the building completes a full analysis of its AYP report, OSAT data, and other student performance indicators. • The goals and strategies of the School Improvement Plan support the goals of the district’s Continuous Improvement Plan. • Other areas are in the evaluation plan document.

29. DATA ProjectAnalysis Plan: Level of Use

30. DATA ProjectAnalysis Plan: Level of Use Change

31. DATA ProjectAnalysis Plan: CIP/SIP Quality Level of use ??? OAKS Data

32. DATA ProjectAnalysis Plan: Sampling of High and Low CIP/SIP Quality Level of use YES OSAT Data

33. The Oregon DATA Project Direct Access to Achievement www.oregondataproject.org