Avoidable Losses Rice University Creekmore Symposium April 29, 2008

1. Avoidable LossesRice UniversityCreekmore SymposiumApril 29, 2008

2. The Problem • 135,000 youth are lost from Texas high schools every year prior to graduation. • These are disproportionately African-American, Latino, and English Language Learners

3. High Stakes Accountability • More than a decade of reform • Aimed at: • increasing student achievement • closing the achievement gap • Became the model for the nation

4. Avoidable Losses: Our Findings • The high stakes accountability system is directly connected to the severity of the dropout problem • This occurs when administrators are in compliance with the accountability system • School ratings go up as weak students exit the system • Students are viewed as assets or liabilities to the school ratings

5. How did we discover this connection? • Data from 271,000 students over 7 years. • Analysis from inside 7 schools • Multi-year case study of a school working to comply with the accountability system • Student interviews to learn their perspectives on life in school under this system

6. Avoidable Losses: The Study • Accountability Miracle in Brazos City • Educate or Comply • The Culture of Accountability: Setting Kids Up to Fail

7. Accountability Miracle in Brazos City Julian Vasquez Heilig, Ph.D. University of Texas at Austin

8. Texas Miracle • In 1993, the Texas Legislature enacted a ground-breaking statute that mandated the creation of the Texas public school accountability system to rate school districts and evaluate campuses • During the 2000 presidential campaign, George W. Bush touted the “Texas educational miracle” as a model for the rest of the nation • George W. Bush and Rod Paige, two primary arbiters of NCLB, lassoed their ideas for federal education policy from Texas

9. The Miracle • Closing achievement gap • Low dropout rate • Soaring Exit test scores • Rising graduation rates • Rising accountability ratings

10. Closing the Achievement Gap?

11. BCSD Reported Dropout by Year

12. Graduation Rates and Exit • BCSD reported: Graduation rate had soared 21 percentage points over five years, from 54 percent in 1997 to 75.3 percent by 2002 • Local Newspaper characterized the high school achievement gains with the headline: “Sophomores soar in Exit TAAS exam”

13. TEA Accountability Ratings for BCSD High Schools (1996-2002)

14. TEA Accountability Ratings for BCSD High Schools (1996-2002) • Considering BCSD’s closing achievement gap and rising accountability ratings (ultra low dropout and high graduation rates) it might be suggested that BCSD schools dramatically improved urban minority students’ academic and enrollment success. • Indicative of an education miracle- worthy of national acclaim and replication into a comprehensive national educational policy (NCLB).

15. Accountability Miracle Reconsidered • Testing analysis • Longitudinal cohort analysis • Grade retention • Graduation

16. BCSD Testing: High-stakes vs. Low-stakes • On average, about 12% of African American and Latino elementary and middle school students were excluded from the TAAS (high-stakes) but not Harcourt (low-stakes) testing from 1997-2002. • In contrast to the TAAS, there is little difference in test-taking patterns on Harcourt tests by race/ethnicity. • As a result, greater proportions of Latino and African American students’ test scores were excluded from TEA accountability ratings

17. Mean Scores by Participation Those excluded from the English TAAS, the basis of the state and district accountability rankings, scored significantly lower on the Stanford-9 than those who took the TAAS across all racial/ethnic groups (p<.001)*. *A GLS regression analysis examined the effects of a dummy variable representing inclusion or exclusion in the English TAAS (controlling for race / ethnicity, language status, income status, and school-level proportions of at-risk students).

18. BCSD High School Cohort Progression

19. BCSD High School Cohort Progression: Latinos and African Americans

20. BCSD Grade Retention by Year

21. BCSD 9th Grade Retention by Race/Ethnicity

22. BCSD 1997-2001 Cohort Testing on the Exit Retention?: Only 209 of 3,489 retained students (about 8% of the total) ever became eligible to graduate by passing all three subjects on the spring Exit TAAS.

23. BCSD Graduation Rate of 1997-2001 9th Grade Cohort (5-Yr) • Large number of students are being retained and otherwise not advancing with their peers through school • Many students not testing on the Exit • Cohort graduation rates should be correspondingly low

24. BCSD Graduation Rates • Vast discrepancy between the cohort graduation eligibility and status rates and publicly released rates • District hemorrhaged students, especially between the 9th and 10th grades • A counselor at King high school estimated from her experience, “I would think that the graduation rate is closer to 40-45%, not 85%.” She related, Ultimately what’s happening is that we’re letting kids down. We’re using some kind of system to disguise where they are. If you’ve got 600 kids in your school and 300 graduate….they’re somewhere...

25. Left Behind? • Is it possible that high-stakes testing and accountability may not be as good as advertised? • Voices from BCSD high schools on their response to the pressure of high-stakes testing and accountability

26. Inferential Regression Analyses • Do actions that impact student progress spur increases in high-stakes test scores and TEA accountability ratings? • GLM Random-Effect Regression • Increasing retention and disappearance of students increases average Exit scores • Multinomial Logistic Regression • By increasing 9th grade retention, schools increase and maintain accountability rating • Robustness of GLM random-effect and multinomial logistic regressions suggest that manipulating student progress can increase Exit scores and accountability ratings

27. Summary of Findings • High-stakes test scores and accountability ratings steadily rise as students steadily leave • Incentives to game and reduce educational opportunity for low-performing students • High-stakes testing and accountability policies led to an escalation of deleterious outcomes disproportionately impacting minority students

28. To Comply or To Educate Judy Radigan , Ph.D.

29. A Case Study in Compliance • Edgeview H S, representative of Brazos City high schools with losses of low-income and minority youth • 75% Latino, 20% African American, 4% White, 10% English Language Learners • Reforming school working to personalize the learning environment with small learning communities • Spring 1997 – Principal targeted to improve scores markedly or face TERMINATION

30. Ninth Grade Waiver • Based grade promotion from 9th grade on passing of core courses: English, social studies, math, science rather than number of credits gained • All the schools had it. In fact, we were one of the last schools to accept the waiver because philosophically we thought [to do so] would be cheating. We were testing everybody. We felt we had no choice but to move to that waiver in order to save face and get our scores up. • A lot of the kids were defeated, and they dropped out.

31. Texas Miracle Edgeview’s accountability rating moved from “acceptable with acceptable progress” to “recognized with exemplary progress.”

32. Withdrawal from High School

33. 100 90 80 Proportion Retained 70 60 1997-1998 1998-1999 50 1999-2000 2000-2001 40 30 20 10 0 Fine Oak Edgeview King Carver Clearbend Crockett Douglass Patterson 9th Grade Retention

34. Structural Change Small Learning Communities

35. Accountability Incentives: From System to Principal Federal State vendors consultants District school school school school school

36. Texas Miracle? It’s not a miracle to manipulate things. A miracle is saving kids actually, in reality—that’s what miracles are. It’s not to manipulate things so that it appears—it’s a facade.

37. The Culture of Accountabilty:Setting Up Kids to FailEileen M. Coppola, Ed.D.

38. Expanding the Lens/Drilling Down into Student Experience • How does what we’ve seen at Edgeview become systematically embedded? • How does the operation of the accountability system become “normal” or unquestioned? • How do students experience school under this system? • How does their experience affect their motivation and ability to stay in school?

39. Data • Focus groups and individual interviews with school staff and students in 7 Brazos City High Schools. 3 schools majority African-American; 4 majority Latino; all majority low SES. • 14 Administrators, 24 Math and English Teachers. • 122 students selected by teachers from senior English classes and volunteers from a program for former dropouts.

40. Other Components of the Accountability System • Zero-tolerance attendance policy. • Grade retention. • Curriculum degradation.

41. Beneath the Surface: Connecting Dropping Out with the Accountability System • Extensive grade retention leads to frustration. • Texas’ 90% rule on attendance mean that students are penalized and lose credit for courses. • The demands of high-stakes testing degrades the curriculum over time. • Punitive culture and rules conflict with cultural norms and adult status of students.

42. Grade Retention Considerable research shows an increased likelihood of dropping out each time a student is retained in grade. High levels of frustration, disorientation, and alienation reported by students, leading to other choices than continuing school.

43. The Effects of Retention (Focus group with former dropouts who returned to school) Jose: I’ve repeated ninth grade four – four times. Arturo: Same thing with me. I repeated ninth grade four years because I just slacked off. I was lazy and – Int: You repeated ninth grade four years? Arturo: Yes. …Well part of my problem was my freshman year it was – if I didn’t know anything I wouldn’t do it, or I wouldn’t go to class because, you know, I was afraid if they’d call on me and have me go up I wouldn’t know how to work it so I just wouldn’t go to class. So that was just maybe the reason.

44. The Effects of Retention, Continued Paolo: Oh, yeah, they had me taking Algebra for, like, three years straight. I passed the first year, so in the second year I just decided not to go. I tried to get it fixed, but they wouldn’t fix it. So after the third week trying to get it fixed I just stopped going. Int: So did they count you as absent? Paolo: Yeah, and when they tried doing that I just got more aggravated and stopped coming to school period.

45. Attendance Policies • 90% rule • Loss of credit • Evolution over time/credit recovery courses • Intervention of Justice of Peace and Juvenile Court – ticketing • Yet, many reasons for attendance issues: family, working, need for flexibility.

46. The Effects of Attendance Policy (Focus group with former dropouts who returned to school) Carla: I went to three elementary schools…then Middle School. Then I came to ninth grade. When I got to ninth grade, you know, I just made it as a fun thing, you know. I had more freedom than what I had in other schools. Learned – made new friends, you know. Well I flunked ninth grade and back to ninth grade again. Kept going in circles and circles, and my mom used to get tickets and tickets so, you know, it’s just left for me to drop out instead of me just giving my mom nothing but tickets. So I dropped out, and I stayed, like, a whole year without being in school. Int: So what did they actually say – I mean, what happened?  Carla: They sent me a truancy letter saying that if I missed one more day of school that I would be fined….A $500 ticket and a court date.

47. Effects of Attendance Policy, Continued Lavone: Per absence. Carla: Per absence, and I was – but I had my excuse. But most students – Int: See, I was wondering if that $500 fine, if they would (that would make you really stop coming to school? Lavone: That would be a lot. Sean: Some people – rich people really don’t care. For that reason they know if they’re still going to miss another day of school they’ll probably okay with it. Carla: I might as well drop out. Sean: I might as well drop out.  Carla: And no one’s going to pay $500 a day.

48. Curriculum Degradation • Curriculum that is increasingly fragmented and shaped by the forms and priorities of the standardized tests themselves. • This fragmentation leads to inauthentic curriculum that further alienates students, who become bored and frustrated. • Schools continued to have limited capacity to help struggling students. Students complain that they struggle, and worry about the tests, but cannot understand the work.

49. Example: Curriculum Degradation Irma: Instead of teaching us the real life things that we going to need for college and stuff, they started zeroing in just on that test. So it makes everybody nervous, and it threw everybody off. So, like our curriculum is thrown off, ‘cause what they originally were teaching us in the subjects, all of the sudden they switched and then they were just zeroing into this test.

50. Second ExampleCurriculum Degradation Fernando: Math is – its not that it’s boring, it’s just hard. I’ll get it in the beginning, and I’ll get lost. I won’t know what’s going on…[The teacher] helps but, you know, it just – John: He [the teacher] doesn’t break it down a lot. Fernando: Yeah, he don’t – and then his examples are wack. He takes it out of his mind. Int: How does he decide what math you guys should learn? Fernando: Right now he’s teaching us the TAKS. John: Yeah, right now he’s preparing us for TAKS…. John: They teach us what’s on the test – what’s going to be on the test. Fernando: You need that to graduate.