Incident Data: The Tower of Babel

1. Incident Data: The Tower of Babel NCES MIS Winter 2012 Conference Sonya Edwards and Justin Lane

2. Topics • Background – how we got here • Demonstrate the need to bring clarity to discipline data collections and reporting requirements • Show that the Common Education Data Standards (CEDS) is a useful reference tool – establishes a common data vocabulary • Highlight where CEDS falls short of education data stakeholders needs

3. Babel Needs Some Discipline – 2 minute exercise NCES Handbook Incident

4. Building the Tower of Babel • EDFacts discipline reporting requirements • Incident; • Students disciplined • Legacy summary level data collection only met students disciplined requirement • Designed/developed student-level data collection to meet both • Need contingency for LEAs who couldn’t get student-level data in during first year • Tried to modify legacy summary data collection to include Incident data • Arrived at Babel

5. EDFacts Discipline Data Requirements N030 - Discipline Incidents Definition of Discipline Incident (DG 523): The cumulative number of times that students were removed from their regular education program for at least an entire school day for discipline. DG 523 falls short of adequately defining “Incident” The term “cumulative number of times” is vague. Does it mean how many unique times student’s offense led to discipline? What if they were disciplined under the same category but at different times? Also, EDFacts definition is not consistent with the NCES or Dept. of Justice definitions that uses phrases such as “single occurrence” or “same event.”

6. EDFacts Discipline Data Requirements N136 - Students Disciplined Definition of Students Disciplined (DG 673): The number of students removed from their regular education program for at least an entire school day for discipline.

7. Summary Level Data Collection for Discipline Data Excerpt from draft design of our summary level collection to meet the requirements of EDFacts files N030 and N136. Note: this form doesn’t collect grade level data required in N136

8. Researched How It Could Be Done • Polled other states on edit checks • Researched National Data Standards • CEDS, NEDM, NCES Data Handbooks • Got even more confused #%)*&%

9. Polled Other StatesOn Edit Checks We're looking at building the following edit checks into our data collection for incident data. Just wondering if these make sense to those of you in other states. We’re definitely not sure about number 4. 1. The number of incidences (most severe) cannot be greater than the number of students who have committed that offense. 2. The number of incidences (most severe) cannot be greater than the number of times offense occurred. 3. The number of times offense that occurred cannot be greater than the combined number of expulsions and suspensions. 4. The number of students that committed that offense cannot be greater than the number of times offense occurred.

10. Responses from Other States 1. The number of incidences (most severe) cannot be greater than the number of students who have committed that offense. Responses: • State A: What do you mean by the number of times? Is this not the same as an incident? Our thought: Probably not • State D: The number of incidents can be more than the number of students when there are unknown offenders (such as anonymous bomb threats, fire-setting, gun found in a bush). Our thought: Okay, but EDFacts N030 requires reporting on when a student is removed for at least an entire school day. Are other states collecting data that doesn’t result in suspensions or expulsions?

11. Responses from Other States 2. The number of incidences (most severe) cannot be greater than the number of times offense occurred.  Responses: • State D: OK Our thought: given we only had one state respond, did they understand what we were asking?

12. Responses from Other States 3. The number of times offense occurred cannot be greater than the combined number of expulsions and suspensions. Responses: State B: Only makes sense if you are only collecting incident data which results in suspension or expulsion. Our thought: We agree. State D:There should be no relationship between offenses and disciplinary actions. A student may be involved in several offenses (fighting, vandalism, threats to staff) and receive just a single suspension.  Conversely, a single offense can result in an immediate suspension, followed by an expulsion after a board hearing. Our thought: Probably right.

13. Responses from Other States 3. The number of times offense occurred cannot be greater than the combined number of expulsions and suspensions. Responses: • State E: Other states, like us, collect data by student, so if 3 students were involved in a specific event that led to multiple disciplinary actions, it would be reported as 3 different incidents in our system. In our system it is possible to have multiple removals for the same incident/offense.  Our thought: Given they report a single event as different incidents, not sure if they can meet the requirements of EDFacts N030. Are they using incident/offense interchangeably? Do they really mean offense when they use “incident”?

14. Responses from Other States 4. The number of students that committed that offense cannot be greater than the number of times offense occurred. Responses: • State B: What happens if more than one student is involved in an incident (i.e., a fight involving 2 students, both of whom were suspended as a result). • State D: OK, but not sure of the use of this edit check. • State E: The way data systems are set up to collect discipline data varies by state, some collect data by incident and attach students to the incident. If that is the case with your system then this rule would not make sense to me. If two students were in a fight and both were associated with the same incident, I would expect the number of students to be greater than the number of incidents.  Our thought: OK, it definitely looks like there’s confusion about “offense” versus “incident”, etc. Feels like the Tower of Babel.

15. Revisited Requirements and National Standards EDFacts File Spec N030 Discipline Incident CALPADS Data Guide Incident U.S. Dept. of Justice Incident

16. Entity Relationship Diagrams – A Brief Explanation A diagram used to model the relationships between entities. Simsion, Graeme and Graham Witt Data Modeling Essentials, 3rd Ed, p75 Entity (classes) – “categories of things of interest to the business; represented by boxes…and generally implemented as tables.” (p75) Or, “a class of things [including events] we need to keep information about” (p76) Relationships – “represented by lines with crows’ feet [in a conceptual model – arrows in a logical model], and generally implemented through foreign keys”

17. Entity Relationship Diagrams – A Brief Explanation (cont.) • In education, entity types of interest: • Things: student, school, school district • Events: student enrollment, student exit • Classifications: gender category, school type • Relationships and Cardinality • The direction and logic of how entities relate to each other: • One to many: A school can have many enrolled students • Many to many: A teacher can be assigned to many course sections • One to one: There can only be one disciplinary action reported for each student offense (state reporting requirement)

18. CDE Discipline Data Model Incident Student Disciplinary Action Offense Severity Offense Category Student Offense White = Reference Data Student

19. National Education Data Model CA: Student Offense CA: Incident CA: We don't collect and not required in EDFacts Source: Vince Parades Disciplinary Action 1 1 Student Perpetrator Offense Severity many Offense Category 1 Discipline Incident 1 many Victim

20. Data Models: Different Perspectives • California’s • SLDS Data Collection • Only data needed to meet federal and statewide reporting requirements • Only data related to perpetrator (when disciplined) • NEDM • LES SIS perspective?

21. Issue Designing Discipline Summary Level Data Collection When trying to design the summary level collection for discipline data, we discovered it becomes impossible to validate integrity of data because relationships are comingled in summary level data.

22. Discussion Questions • How many of you are aware of the Common Education Data Standards (CEDS)? • If so, how many of you…. • Were actively involved in reviewing and commenting on CEDS 2.0? • Are referencing CEDS, NEDM, or the NCES Handbook as you develop data element names and definitions for your data elements? • Are using the names and/or definitions as they appear in CEDS, NEDM, or the NCES Handbook?

23. Discussion Questions (cont.) What challenges have you encountered in using these national standards? Given that many states have already developed their data dictionaries, what strategies do you think could be taken to move SEAs towards a common vocabulary?

24. Incident Definitions However, CEDS falls short, as it doesn’t appear to provide a common definition of “Incident.”

25. CEDS: Useful Tool The Common Education Data Standards (CEDS) is a good foundation to build a disciplined tower and is a useful reference. For “Incident,” it provides definitions for all the attributes of an “Incident:” • Incident Description • Incident Behavior • Incident Cost • Incident Injury Type • Incident Reporter Type • Incident Identifier • Incident Date • Incident Time • Incident Time Description Code • Incident Location When entities are not defined, risk of confusion increases dramatically.