Using Early Warning Indicators to Identify Students at Highest Risk of Dropping Out

1. Using Early Warning Indicators to Identify Students at Highest Risk of Dropping Out Ruth Curran Neild Center for Social Organization of Schools, Johns Hopkins University

2. Relationship to college and career readiness • Earning high school diploma is a critical juncture in the pipeline • Provides a conceptual model for tracking likelihood of young adult success in postsecondary education and/or work

3. The Checklist Manifesto

4. All professions in the 21st century face increasing complexity in their work

5. The Modern Challenge: Keeping Track of a Lot of “Moving Pieces” • It’s easy for well-trained professionals to: • Forget • Fail to communicate • Overlook • Fail to “connect the dots”

6. Most errors of are those of ineptitude Mistakes that occur because we do not make proper use of what we know (As contrasted with errors of ignorance – mistakes that arise from not knowing what to do)

7. Like other professions, teachers operate in an environment of increasing complexity • Increased responsibility for outcomes of all students, including those who are disengaging from school • Increased responsibility to “individualize” education - to find the “right solution” or “right fit” for each student • Substantial amounts of data about students collected and made available to teachers

8. Early Warning Indicator Systems enable teachers to: • Use empirically-developed data indicators that are most predictive of a given outcome as a “flag” that a student are in trouble • Track interventions that have been assigned to particular students • Systematically track associations between interventions and outcomes for students at their school

9. Logic of Early Warning Indicators Of High School Dropout

10. There are many underlying reasons for dropping out of school • Social science research using secondary data sets has helped us to understand correlates of dropping out and…most importantly, that… • Dropping out is the culmination of a gradual process of disengagement from school

11. For educators, this research has real limitations How do we know which specific individuals are most likely to drop out so that we can target interventions to them? How early in students’ careers can we reliably identify those on the path to dropping out? Can we identify students with readily-available data or do we need specialized assessments?

12. Developed in the Context of “Dropout Factory” Schools • At these schools, 40% or more of the students fail to graduate • Family income, family structure, race/ethnicity, scores on nationally-normed tests are usually the same or within a narrow band in these schools

13. Four questions about EWIs • What are the characteristics of a good EWI system? • What are the signals? • What technological and organizational infrastructure is needed to “capture” the signal? • What can schools and districts do once the signals are identified and captured?

14. Characteristics of a good EWI system Empirically developed: The “signals” are identified through analysis of longitudinal data for prior cohorts of students. High accuracy: A high percentage of students with the “signals” drop out. Conversely, a low percentage of students without the “signals” graduate. High yield: These “signals” capture most of the dropouts (avoiding the “1% problem”). Accessible data: Data that provide the “signals” are readily available and relatively inexpensive to access.

15. How did we identify the “signals” of eventual dropout? • Empirical analysis of cohorts in Philadelphia, starting with 6th graders (Balfanz, Herzog, & MacIver), and 8th graders (Neild & Balfanz, 2006) • Data scan of longitudinal student record data • Test scores • Report card grades • Attendance • Special education and ELL status • Gender • Age • Race/ethnic background

16. Looked for a 75% threshold – why? • Choosing a “strong signal” – students who are at highest risk of dropping out • By not making the net too broad, scarce resources can be targeted at those students who are greatest risk

17. The Big Four in 6th grade • Failing Math • Failing English • Attendance <80% • At least one poor behavior mark (Balfanz, Herzog, & MacIver)

18. 8th grade warning signals • Three factors gave students at least a 75% probability of dropping out: 1. Failing math in 8th grade 2. Failing English in 8th grade 3. Attending less than 80% of the time

19. 54% of the dropouts sent one or more of these signals in 8th grade

20. Had an 8th grade “signal” Did not have an 8th grade signal: Passed 8th grade English Passed 8th grade Math Attended at least 80% of the time

21. 9th Grade signals • Three factors gave students at least a 75% probability of dropping out: 1. Earning fewer than 2 credits 2. Not being promoted to 10th grade 3. Attending less than 70% of the time

22. 80% of the dropouts sent one or more of these signals in 8th or 9th grade

24. Technological Infrastructure: Real time data

25. Conceptual frame for intervention Whole school interventions More labor intensive More specialized More costly Targeted Interventions Intensive Interventions

26. Organizational Infrastructure “Near-peers” to nag and nurture TEAMS of Teachers, ideally all teaching the same group of students Supported by… Links to social services

27. Implications for College and Career Readiness Possibility of using indicators across systems to address readiness EXAMPLE • Connecting school district and local college data to identify high school predictors of key postsecondary outcomes, such as: • Placement out of remedial courses • Overall credit accumulation and in key areas • Return for a second semester or a second year

28. EXAMPLE New York City

29. Implications for College and Career Readiness Possibility of using indicators across systems to address readiness ? There is a great deal that is unknown about whether there are readily accessible, high accuracy, high-yield high school predictors of postsecondary outcomes

30. Implications for College and Career Readiness Possibility of using indicators within a higher education system to identify students at-risk of course failure EXAMPLE • Survey data about study habits in high school and other non-cognitive predictors, combined with data on class attendance and interim grades

31. The Checklist Manifesto The purpose of a good checklist is not to fill out paperwork or to prove to others that we’ve “covered our bases,” but to help well-trained professionals cope with the complexity and detail of their work in the modern world. EWI System teachers keeping students on track

32. Ruth Curran Neild Center for Social Organization of Schools Johns Hopkins University rneild@csos.jhu.edu