Using Early Warning Systems to Target Tiered Interventions for Dropout Prevention

1. Using Early Warning Systems to Target Tiered Interventions for Dropout Prevention An introduction to the Virginia Early Warning System (VEWS) Tool and Implementation Strategies www.betterhighschools.org

2. Virginia Context Keith Smith, Ph.D.Appalachia Regional Comprehensive Center www.betterhighschools.org

3. Virginia Data Data taken from the Virginia Department of Education’s State-Level Cohort Report, Four Year Rate for the Class of 2009 See:http://www.doe.virginia.gov/statistics_reports/graduation_completion/cohort_reports/index.shtml www.betterhighschools.org

4. Background Information In February, 2009, the Board of Education strengthened Virginia’s accountability program by: • Requiring high schools—beginning with the 2011-2012 school year—to meet an annual benchmark for graduation. • Revising Virginia’s accreditation standards to create a graduation and completion index for high schools www.betterhighschools.org

5. Graduation and Completion Index The Graduation and CompletionIndex awards: • 100 points for students who graduate with a(n) • Advanced Studies Diploma • Standard Diploma • Modified Standard Diploma • Special Diploma or General Achievement Diploma • 75 points for students who earn a GED • 70 points for students still in school • 25 points for students who finish high school with a Certificate of Completion www.betterhighschools.org

6. VEWS Pilot Team A team of school divisions, education organizations, and VDOE consultants assisted the VDOE with the pilot of the VEWS Tool • School divisions: Franklin City, Lee County, Richmond City, and Pulaski County • Organizations :Appalachia Regional Comprehensive Center and the National High School Center • Development of an implementation guide based on the pilot divisions experience www.betterhighschools.org

7. Using Early Warning Systems for Dropout Prevention Susan Bowles Therriault, Ed.D. National High School Center at the American Institutes for Research www.betterhighschools.org

8. Who We Are Content Centers (5)including theNational High School Center OSEP TA&D CentersincludingNDPC-SD, NSTTAC, PSO IES Regional Labs 16 Regional Comprehensive Centers (RCCs) 6 Regional Resource Centers (RRCs) State Education Agencies (SEAs) Local Education Agencies (LEAs)

9. Sobering Statistics… Balfanz, R. and Legters, N. (2006, July 12). The graduation rate crisis we know and what can be done about it. Retrieved online from http://web.jhu.edu/CSOS/graduationgap/edweek/Crisis_Commentary.pdf Nearly one-third of all high school students leave the public school system before graduating 1.2 million students drop out of high school each year – that’s 12 million over the next decade 7,000 students drop out of high school every day 15% of the high schools in the U.S. produce 50% of our dropouts – schools Balfanz and Legters call “dropout factories” www.betterhighschools.org

10. Economic Consequences A new high school dropout in 2000 had less than a 50% chance of getting a job That job earned less than half of what the same job earned 20 years ago Lack of education is strongly correlated with welfare dependency and incarceration Cutting the number of dropouts in half would reap $45 billion in revenues and decreased costs (Levin et al., 2007) www.betterhighschools.org

11. Early Warning Systems Early warning systems (EWS) rely on readily available data housed at the school to: • Predict which students are at-risk for dropping out of high school • Target resources at the school and district level to support off-track students while they are still in school, before they drop out • Examine patterns and identify school climate issues that may contribute to disproportionate dropout rates at a subset of high schools or within subpopulations of students www.betterhighschools.org

12. Virginia’s Pre-High School Index The Virginia’s Pre-High School Index uses the following factors to measure risk for students entering the ninth grade: • SOL scores in reading and mathematics, grade 8 • Final course grades in English and mathematics, grades 7 and 8 • Age at enrollment in 9th grade • Retained in ninth grade • Absenteeism • Number of in- and out- of school suspensions www.betterhighschools.org

13. VA Pre-High School Index *Based on students who were enrolled in 9th grade in 2005-2006 in two Virginia school divisions. Outcomes as of February 2010. www.betterhighschools.org

14. 9th Grade is a Critical Year Herlihy, C. (2007). State and district-level supports for successful transition into high school. Washington, DC: National High School Center. Allensworth, E., & Easton, J.Q. (2007). What matters for staying on-track and graduating in Chicago Public High Schools: A close look at course grades, failures and attendance in the freshman year. Chicago: Consortium on Chicago School Research. Neild, R.C. & Balfanz, R. (2006). Unfulfilled promise: The dimensions and characteristics of Philadelphia’s dropout crisis, 2000-2005. Philadelphia Youth Network, The Johns Hopkins University, and the University of Pennsylvania. • Ninth grade is a “make or break year” • More students fail 9th grade than any other high school grade • A disproportionate number of students who are held back in 9th grade subsequently drop out • Monitoring students’ progress throughout 9th grade—and even during the first semester—provides powerful indicators that can predict whether students will complete high school: • Engagement • Course performance • Chicago’s “On-Track” Indicator www.betterhighschools.org

15. Key Indicators Engagement • Attendance/absenteeism Course Performance • Course grades • Number of credits earned “On-track” Indicator • Core course performance & accumulated credits Research from several U.S. school districts provides a strong foundation for defining 9th grade warning signs that students might drop out, but local adaptation is key www.betterhighschools.org

16. “High-Yield” Academic Indicators: Attendance www.betterhighschools.org

17. “High-Yield” Academic Indicators: Course Failures www.betterhighschools.org

18. “High-Yield” Academic Indicators: GPA www.betterhighschools.org

19. Chicago’s “On-track” Indicator Students are “on-track” if they: have not failed more than one semester long core course, AND have accumulated enough credits for promotion to the 10th grade. www.betterhighschools.org

20. On-Track Indicator Source: Allensworth & Easton (2005) www.betterhighschools.org

21. “High Yield” 9th Grade Indicators www.betterhighschools.org

22. Virginia Early Warning System Tool www.betterhighschools.org

23. VEWS Tool • Key Components • Pre-High School Index validated in Virginia Divisions • Progress monitoring of high school students over the course of the school year • Student interventions and tracking • Student level reports • School level reports www.betterhighschools.org

24. VEWS Tool VEWS Tool features enable users to: Upload student data Create student- and new school-level reports Track student interventions by tier and type Add additional and/or custom indicators (e.g., behavior, pre-high school) Modify the benchmarks based on analysis of longitudinal data www.betterhighschools.org

25. VEWS Tool Demonstration www.betterhighschools.org

34. Implementing the Virginia Early Warning System (VEWS) Implementation Process VEWS Guide www.betterhighschools.org

35. Implementing the VEWS Process www.betterhighschools.org

36. Step One: Establish Roles and Process VEWS teams should include school- and district-level individuals who have: • Authority to make decisions • Knowledge of diverse students • Expertise to manage and analyze data • Information about strategies www.betterhighschools.org

37. Step Two:Use the VEWS Tool • Routinely available data are good predictors of whether a student is likely to drop out of high school • Pre-high school indicators—obtain the data before the school year begins • First-month absences, in addition to end of the semester grades, are additional strong predictors of dropout • The goal is to effectively and efficiently allocate dropout prevention resources to change the odds for students with a high propensity to drop out • Not all students at-risk will need all interventions available • Analysis of different patterns of risk can help target appropriately • EWS data = knowledge to make these decisions www.betterhighschools.org

38. Step Three: Analyze VEWS Data Questions about EWS data: • Student-level patterns: What do your data tell you about individual students who are at-risk? • School-level patterns: What do your data tell you about how the school is doing? • Are students who were flagged from the beginning remaining “off-track” through the year? • Are students who were flagged at one reporting period back “on-track” at the next? www.betterhighschools.org

39. Step Four: Interpret VEWS Data Digging deeper than the indicators: Indicators are just observable symptoms, not root causes Characteristics of students who are flagged can further help target interventions at the appropriate intensity www.betterhighschools.org

40. Step Four: Interpret VEWS Data (cont.) Understanding Characteristics of Students At-Risk Decisions to persist or drop out are affected by multiple contextual factors - family, school, neighborhood, peers. Personal and school factors contribute to success or failure during the freshman year Attendance and course performance problems are distinct indicators in the VEWS but are highly interrelated, and both can signal disengagement Student background characteristics are less important in explaining failures than behaviors in high school www.betterhighschools.org

41. Step Five:Applying Interventions Example Programs that incorporate this strategy: America’s Choice First Things First School Development Program Talent Development High School Example Programs that incorporate this strategy: America’s Choice Talent Development High School Example Programs that incorporate this strategy: America’s Choice Check and Connect Coca-Cola VYP Interpersonal Relations Personal Growth Class NGP Quantum Opportunities Program School Development Program Talent Development High School Twelve Together Example Programs that incorporate this strategy: Academic Literacy Program Positive Behavioral Interventions and Supports (PBIS) Rehabilitation, Empowerment, National supports, Education, and Work (RENEW) RTI Strategic Instruction Model Example Programs that incorporate this strategy: ALAS Check and Connect Coca-Cola VYP Interpersonal Relations Personal Growth Class PBIS Project COFFEE Talent Development High School Teen Outreach Program Example Programs that incorporate this strategy: Middle College High School NGP STEP Example Programs that incorporate this strategy: ALAS Check and Connect First Things First Interpersonal Relations Personal Growth Class Project COFFEE Twelve Together Example Programs that incorporate this strategy: Career Academies First Things First Middle College High School NGP Project COFFEE Talent Development High School Example Programs that incorporate this strategy: Project GRAD Example Programs that incorporate this strategy: Career Academies Ninth Grade Success Academies STEP Example Programs that incorporate this strategy: Check and Connect PBIS Example Programs that incorporate this strategy: Career Academies Learning to Work Program (NYC) Lifelong Options Program (LOP) Middle College High School Project COFFEE RENEW Twelve Together Example Programs that incorporate this strategy: ALAS First Things First Support Center for Adolescent Mothers (Family Growth Center) NGP Talent Development High School Example Programs that incorporate this strategy: ALAS America’s Choice Career Academies Check and Connect Support Center for Adolescent Mothers (Family Growth Center) Middle College High School NGP Quantum Opportunities Program Teen Outreach Program Focus on achievement in core courses Content recovery courses Tutoring as an academic support Tiered approaches Attendance and behavior monitors Advisories and team teaching Counseling and mentoring Small learning communities and school within a school for greater personalization Partnerships between high schools and feeder middle schools Ninth grade transition programs Support for students with disabilities outside of school Career and college awareness Family engagement Community engagement

42. Tiered Approachto Dropout Prevention www.betterhighschools.org

43. New Hampshire's Model Examples: RENEW (Rehabilitation, Empowerment, Natural supports, Education and Work) facilitators Examples: Intervention Team of specialists and administrators will focus on students who exhibit challenging behaviors and who are at risk Examples: Positive Behavioral Supports model (PBS) and Universal Leadership Team including a diverse representation of “opinion-leaders” www.betterhighschools.org

44. Minnesota’s Model Examples: Individualized behavior plans, wrap-around services, alternative programs (5% of students) Examples: Interventions build specific skills, school-within-a-school model, or mentoring programs (15% of students) Examples: Extracurricular activities, school-to-work programs, or positive discipline programs www.betterhighschools.org

45. Step Six: Reflect and Revise During the school year: Regularly, collectively, and systematically Over multiple school years: Validate the indicators to maximize predictive power of the system, e.g. www.betterhighschools.org

46. Questions about EWS? National High School Center Dropout Prevention Resources • Dr. Susan Bowles Therriault, National High School Center at the American Institutes for Research • Email: stherriault@air.org • Phone: 978-371-8321 • Website: www.betterhighschools.org/topics/DropoutPrevention.asp VEWS Data and technical questions • Dr. Deborah Jonas, Executive Director for Research and Strategic Planning, Virginia Department of Education • Email: Deborah.Jonas@doe.virginia.gov • Phone: 804-225-2067 VEWS School improvement and intervention questions • Dr. Kathleen Smith, Director, Office of School Improvement, Virginia Department of Education • Email: Kathleen.Smith@doe.virginia.gov • Phone: 804-225-2865 www.betterhighschools.org