The Future of Education: Creating a Culture of Data-Based D ecision M aking

1. The Future of Education: Creating a Culture of Data-Based Decision Making Dr. Cory J. Steiner Data Steward NDASA Round Table—Midwinter Conference State of North Dakota

2. Objectives • By the end of this presentation, participants will: • Understand the role leadership plays for utilizing data in educational organizations • Understand the four frames of ‘reframing organizations’ and roles the frames play in leadership • Understand importance of creating a culture of quality data • Identify roles needed to build a culture of quality data • Understand the State Longitudinal Data System (SLDS) past, present, and future • Understand job responsibilities of Data Steward • Understand practical use of the School Profile, Assessment Inventory, ACT Student Detail, NDSA Growth Model Roster, and NDSA Assessment Trend, Student Directory reports, and Developmental Courses by Subject Area, and District Developmental Courses

3. Purpose • The focus must be on moving from good to great • Get a little better every day • `If you keep doing what you have been doing, you will keep getting what you have been getting • Stockdale Paradox • ‘Retain faith that you will prevail in the end, regardless of the difficulties and at the same time confront the most brutal facts of your current reality whatever they might be’ Good to Great: Why Some Companies Make the Leap and Others Don't (Collins, 2001)

4. Talking Point #1: Job Responsibilities

5. A Data What? A Data Steward… • Provide data administration for ND K-12 State Longitudinal Data System (SLDS) • Facilitate ‘cleaning’ of data (i.e.—garbage in, garbage out) • Provide professional development for teachers and administrators • Present to future teachers and administrators • Develop security course • Develop ethics course • Develop assessment workshop • Develop, lead and support services for K-12 Registrars • Major initiative and development of training program • Participate in strategic planning for the K-12 SLDS • Utilize a Strengths, Weaknesses, Opportunities, and Threats Analysis (SWOT) • Maintain a high level of professional competency • Communication is key

6. Talking Point #2: Building a Culture with Quality Leadership

7. Get Two • Who are the leaders in your organization? • What is one thing an educational leader can do that (if done well) will make a difference in an organization?

8. Roles for Leaders in the ‘Data’ Movement Administrators Teachers 1. 2. 3. 4. 5. 1. 2. 3. 4 5.

9. How Do You Shift Your Culture? • Create A Sense of Urgency • Any successful change starts with urgency • Complacency is common throughout organizations • ‘We are much too complacent…and we don’t even know it’ • Don’t mistake busy for non-complacent • Change is not coming… • It is here • We must pay attention to change, but do not put it on a pedestal • Focus must shift from change to the idea of continuous improvement • Before this can happen, correction must take place • Bottom Line is… • We are paying insufficient attention to key opportunities • SLDS is the opportunity to reframe your organization

10. Reframing Organizations • Four frames of reference in decision-making (for all positions in education): • Human resources (people within organization) • Symbolic (culture) • Structural (procedures & policies) • Political (public) • Every decision you make in your organization has an element from each frame (i.e.—Changing your lunch schedule, school calendar, etc.). • Every decision has an affect on your culture Reframing Organizations: Artistry, Choice, and Leadership (Bolman & Deal, 2003)

11. Data-Driven Culture—A Model • All student achievement decisions are based on data and not on adult preferences • All instructional staff are involved in decisions • All instructional staff members are involved in collaborative teams (PLC’s) that analyze state, district, school and classroom assessment data to: • Plan instruction • Set curricular priorities (strong link with common core) • Develop action plans (with smart goals) • Work towards achieving adequate yearly progress • Engage in program evaluation • On-going support and professional development is provided to refine skills in using data to make decisions that affect students and programs • Value quality data • Value a careful and ethical approach to using and sharing data • Must create a culture that values self-reflection

12. Data Quality • ‘Data pays us and grades us’ (NCES) • Organizations/schools that value quality data engage in the following behaviors: • Challenge stereotypes that are linked to data • Don’t rush to judgment • Don’t abuse the ‘Y’ axis • Don’t use data to punishor create anxiety • Don’t share identifiable information (FERPA)

13. Remember: Rushing to Judgment Has Legal Ramifications Personally Identifiable Information: Other information that alone, or in combination is linked or linkable to a specific student that would allow a person in the school community to identify the student with reasonable certainty

14. Something to Keep in Mind North Dakota’s general rule is: A group’s data, whose size is less than 10, can not be publicly displayed.

15. Truth, Lies & Data Quality • A school created a website mistakenly telling 76 students they had been accepted into the school when in fact they had not been. • The US Government mistakenly sent Christmas ornaments to 1,150 deceased marines and sailors with a flyer about an athletic reconditioning program. • Some students were able to obtain administrative password to the system and logged in and cleared absences /tardies for a fee. • A stolen laptop that contained command codes used to control the international space station was not encrypted. • A high school in Las Vegas was ranked 13th among U.S. News and World Report based upon their student teacher ratio of 4 to 1 and a 100% pass rate on AP exams.

16. Talking Point #3: The State Longitudinal Data System

17. State Longitudinal Data System (SLDS)

18. What is a Data Warehouse? • Logical and strategic ordering and storage of data in central area • System consists of a statewide data warehouse that allows program evaluation over single or multiple years • Integrates data from several state agencies

19. SLDS Grantee States

20. North Dakota Statewide Longitudinal Data System (SLDS) • SLDS is a cooperative project between: • Information Technology Department (ITD) • Department of Public Instruction(DPI) • North Dakota University System (NDUS) • Department of Commerce • Department of Career and Technical Education • Job Service of North Dakota • Education Technology Council (ETC) • Department of Health • Department of Human Services • Elements for education (K-12 and higher education), training, and employment programs • For K-12, provides data for: • Program evaluation • Student evaluation • Student programming (next day availability)

21. Goals of Statewide Longitudinal Data Systems • Evaluate teacher programs to improve instruction • Know if graduates have skills to succeed in postsecondary and/or workforce • Simplify local, state, and federal reporting • Support data-driven decision-making for all educators http://nces.ed.gov/Programs/SLDS

22. Tool for Analyzing Data • How are we doing? • Compared to SelfGrade level, Sub Groups, Trends • 2. Compared to Others • National, State, Similar Schools • Compared to Absolutes • Standards, Cut Scores, Scale Scores, Readiness • Michael Fullan

23. In-Depth Analysis • Creating Information and Avoiding DRIP, • (Data Rich Information Poor) • Drill Down • 2. Go Visual • Export

24. Elements of Longitudinal Data Systems • Student Enrollment Information • Information on Graduates, Transfers, Dropouts • State Assessment Scores • Information on Students Not Tested • College-Readiness Test Scores • A Teacher Identifier System • Student Transcript Information • Data on Student Transition and Success in College • Data on Preparation for Success in Postsecondary Education • An Audit System to Ensure Data Quality • Ability to Share Data from Preschool Through College • Unique Student Identifiers http://nces.ed.gov/Programs/SLDS

25. ND SLDS in Stages • Completed (infant) • Provided data to the Regional Education Associations (REAs) for Hess Grant • Produced matches to Workforce data for Adult Ed and CTE to create federal reports • Modified Vital Statistics system to include birth records in the SLDS • In Progress (adolescent) • Expanding master person index (Active Directory Project) • Gathering requirements for postsecondary data in the SLDS • Developing data pump for Vital Statistics data • Assigning state student IDs at birth • Future (adult) • Review portal products, potential vendor offerings, and the best direction for ND • Gather new report requirements and expand SLDS to gather relevant data • Plan work necessary to fulfill postsecondary grant

26. Phases…Now and in the Future • Training • Phase 1 is ongoing • Introduce SLDS and basic features to administrative assistants, administrators, coordinators, and directors throughout the state • 288 participants • Will be doing another round of initial training • Phase 2 is beginning • How to use SLDS to impact student achievement (‘drill down’ to student level) • ‘SEED’ project with individual schools from each REA (Grafton, Maddock, and Kulm)

27. Talking Point #4: What Really Matters

28. SLDS: An Avenue for Self-Reflection • ‘Use data for good and not evil’ • Data Set #1 • School A • Scored 3 points more per game than previous year • School B • Scored 20 points more per game than previous year • Discussion Question—Get Two • What can you say about this set of data? • Data Set #2 • School A • Scored 70 points per game in previous year • School B • Scored 38 points per game in previous year • Discussion Question—Get Two • What can you say about this set of data? • Discussion Question—Get Two • What other data do you need to say?

29. Questions Districts Must Ask…And Be Ableto Answer…SLDS Provides Answers Attendance Academics What is the percentage of overall attendance? What groups have the lowest attendance rate? What groups show growth? Where is their a gap in achievement? Who is meeting their growth targets and goals? What are the characteristics of groups above, at, or below grade level? What progress monitoring is taking place?

30. SLDS: A Practical Approach to Training • Lead to root cause analysis • Reduce the problem and not just symptom • Reduce wasted efforts and resources • Encourage critical conversations and self-reflection • Reinforce rationale for decisions Root Cause Analysis: School Leader's Guide to Using Data to Dissolve Problems (Preuss, 2003)

31. The Five Whys • Simple problem-solving and identification • Ask the question ‘why’ five times • Dig ‘deeper’ to find a cause • Example: ‘We have too many students tardy to class’ • Why do we have so many tardies • ‘Students are saying they don’t have time to get to their next class’ • Why don’t students have time to get from one class to another? • ‘Students are saying that four minutes is not enough time to get around’ • Why is the passing time only four minutes? • ‘We wanted to reduce the amount of time students were in the halls’ • Why did we want to reduce time students were in the halls? • ‘We were having a lot of issues with student behavior and discipline’ • Why did we want to reduce disciplinary problems? • ‘It was taking us away from our classes and taking students out of our classes’

32. Talking Point #5: Key Reports

33. Five Key Reports

34. School Profile Report • Purpose: Provides an overview of enrollment, attendance, and NDSA results.

36. Assessment Inventory Report • Purpose: List all assessments that are currently available in SLDS.

39. School Assessment Summary Report • Purpose: Provides summary information selectable by assessment and subject area (NDSA, ACT, MAP, AimsWeb). Drills to school assessment details by assessment subject to allow for filtering on program and student demographics.

41. NDSA Growth Model Roster Report • Purpose: Provides an overview of individual students in comparison to meeting their growth index

43. NDSA Growth Model Roster Report: So What, Now What • ‘Goal setting is the single most powerful motivation tool in a leader’s toolkit’ (Blanchard) • Allows you to set SMART goals with students

44. NDSA Trend Report • Purpose: Provides NDSA trend data for a school.

46. Student Directory Report • Purpose: Displays student proficiency details selectable by school year, grade, school, proficiency level, and student demographics

48. Student Dashboard • Will be able to ‘designate’ students as at-risk • Will do so by: • Attendance • Assessment Scores • Course Grades • 10% or more drop in average • Grades below a ‘C’ level • Courses repeating • Failing courses

49. The Student Dashboard

50. New and Improved Student Dashboard