The DATA WISE Process and Data-Driven Dialogue

1. Presented by: Lori DeForest ldeforest@cnyric.org (315)433-2247 The DATA WISE Process and Data-Driven Dialogue

2. “Using data effectively does not mean getting good at crunching numbers. It means getting good at working together to gain insights from student-assessment results and to use the insights to improve instruction.”- Kathryn Boudett, Elizabeth City, & Richard Murnane, “When 19 Heads Are Better Than One,” Education Week, December 7, 2005.

3. The DATA WISE Process and Data-Driven Dialogue

4. Organize for Collaborative Work Prepare • Build Data Teams • Establish team structure to allow for data discussions • Establish norms • Utilize protocols • Complete a Data Inventory

5. Data Teams Prepare • Raise important questions about student learning and achievement • Assist in organizational aspects of data • Dialogue about multiple data sources • Examine and interpret data • Investigate ways to improve teaching and learning

6. Curriculum CouncilsInitial Data Teams Prepare October and November - Setting Norms Protocol - Compass Points Protocol

7. Setting Norms Protocol Prepare What norms do we need? • Brainstorm • Discuss • Synthesize • Build consensus

8. We need to build emotional safety to reach cognitive complexity.B. Wellman and L. Lipton

9. Curriculum CouncilsInitial Data Teams October and November - Setting Norms Protocol - Compass Points Protocol December and January - Data Inventory - Data Analysis Tools

10. Purpose of Data Inventory Prepare • Summarizes all the types of data that are available and helps to determine what other data is needed • Builds assessment literacy • Assists in the planning of using data effectively • Begins conversation about educational questions

11. Race/Ethnicity • English Proficiency • Attendance Data Inventory Prepare

12. Data Inventory Prepare

13. The DATA WISE Process and Data-Driven Dialogue

14. Assessment Literacy Prepare scaled score norm-referenced cohort reliability validity cut score performance levels measurement error raw score grade equivalents sampling standards-referenced percentile rank criterion-referenced

15. Data Analysis Tools • COGNOS Report Net • COGNOS Power Play Cubes • Data Mentor • nySTART • NYS State Report Card Databases and ELA and Math Media Databases • Student Management System Demonstrate tools at Curriculum Councils, Grade Level and Department Meetings Offer training opportunities

16. The DATA WISE Process and Data-Driven Dialogue

17. Inquire Data Overview • Determine audience • Decide on educational questions • Create graphic displays of standardized test results • Engage in conversations around initial data set

18. 2006 English Language Arts Performance Data Sources: CNYRIC COGNOS PowerPlay Cubes, NYSED School Report Card and ELA Assessment Databases

19. 2006 Mathematics Performance Data Sources: CNYRIC COGNOS PowerPlay Cubes, NYSED School Report Card and Math Assessment Databases

20. The DATA WISE Process and Data-Driven Dialogue

21. “Without an investigation of the data, schools risk misdiagnosing the problem.”Data Wise, 2005

22. Inquire Data Analysis Protocol • Activate and Engage • Set norms • Articulate predictions and assumptions • Explore and Discover • Begin with a single data source • First describe what you see • Ask questions • Identify additional data needs

23. The DATA WISE Process and Data-Driven Dialogue

24. Data Analysis • Writers Club

25. Informing Planning for Writers Club Use COGNOS PowerPlay to identify needs of struggling students.

26. A distracter analysis may help you understand children’s incorrect thought processes.

27. Data Analysis • Writers Club • Physical Education program • Intervention Analysis

28. Intervention Analysis 2006 Cohort ELA 8 Performance of students who performed at Level 1 or 2 on ELA 4 and remained in district (n=43) Tracking cohort performance may give you some information about program or student growth.

29. Data Analysis • Writers Club • Physical Education program • Intervention Analysis • English Language Arts Analyses

30. English Language Arts Educational question: Is student performance declining in reading comprehension but increasing in listening comprehension? • Utilize COGNOS PowerPlay item analysis and Scoring Key to classify questions by subtest • Compile data using formulas and functions • Study trends over time • Use comparative data to inform analysis

32. The graph above illustrates the importance of using comparative data to inform analysis.

33. English Language Arts Analyses COGNOS Report Net now houses analysis reports which can provide you information about student performance.

34. English Language Arts Analyses

35. Sampling Principle “Because a test is not a direct measure of a student’s degree of mastery of an entire domain, any conclusion you reach about proficiency in that domain is based on an inference from proficiency on the smaller sample. …Even a test that provides good support for one inference may provide weak support for another.” Data Wise, 2005

36. Data Analysis • Writers Club • Physical Education program • Intervention Analysis • English Language Arts Analyses • Math Analyses

37. Data Overview of NYS Assessment Performance • Considerations when reviewing summary data • Different cohort groups • Different samples of items each year • Different test blueprints • Importance of comparison data sets

38. Go to www.emsc.nysed.gov/osa and the Report Card link to access data for similar schools.

39. Enrollment in College Level Math Courses Note: Dual enrollments taken into account for total annual percentage Utilize your own student management system to analyze additional data as well.