Sunny Becker Human Resources Research Organization June 21, 2013

1. Relationship of High School Exit Exams to Post-High School Outcomes A Collaborative Effort in California Sunny Becker Human Resources Research Organization June 21, 2013

2. California Independent Evaluator CAHSEE Independent Evaluator 2000 Present

3. California Independent Evaluator CAHSEE Independent Evaluator 2002: Census administration to sophomores in Class of 2004 2000 Present 2002

4. California Independent Evaluator CAHSEE Independent Evaluator 2006: Passing CAHSEE becomes graduation requirement 2000 Present 2002 2006

5. California Independent Evaluator CAHSEE Independent Evaluator 2010: Post High School Outcomes (PHO) Study was authorized 2000 Present 2002 2006 2010 PHO Study

6. Research Questions • What post high school outcomes can be linked to CAHSEE performance? • How well and in what ways does CAHSEE performance predict post high school performance? • How feasible is a collaborative effort among volunteer LEAs to analyze the relationships between CAHSEE performance and post high school outcomes?

7. Post High School Outcomes (PHO) • Limited funds • Comprehensive Study Proof of concept • Local education agencies (LEA): school districts and charters • Voluntary participation • Mutually beneficial • Highly collaborative • Brainstorming Workshop • Initial Results Workshop

8. The Challenge: Student Outcomes • Ideal: Follow up with high school graduates • One small charter school did this • Senior Surveys • Good for all outcomes (postsecondary education, military, civilian work, other) • Cost effective • Intentions as proxy for actual outcomes • Some LEAs already administered senior surveys that were used as a basic template for LEAs who did not have them

9. The Challenge: Student Outcomes (cont’d) • Senior surveys (continued) • College enrollment plans, including college characteristics • Career fields • Highest degree • Work full time • Military enlistment by branch of service • and other things.

10. The Challenge: Student Outcomes (cont’d) • National Student Clearinghouse’s Student Tracker (NSC ST) • As of July 2012, more than 3,300 colleges and universities contribute to the NSC database, approximating 96 percent of all students enrolled in U.S. public and private postsecondary institutions. • “No record found” • These students may never have enrolled in a postsecondary institution in the United States, or • The student may be among the 4% of students who enrolled in one of the institutions for which NSC does not collect data, or • The student may have enrolled outside the U.S., or • The student may have requested that their data not be shared, or • The ST matching algorithm may have failed.

11. The Data We Collected • Student demographics • CAHSEE ELA and Mathematics test scores • Early Assessment Program (EAP) scores • Senior Surveys • Asked LEAs to administer a Senior Survey with common items to Class of 2011 • Some LEAs already had senior surveys with no individual ID • NSC ST • Some LEAs subscribed on own • We sampled an additional 20,000 students • Sampled by LEA and graduating class (2007, 2009, 2011)

12. PHO Study: Full Sample & Analytic Sample • 12 LEAs • 114,000 graduates in Classes of 2006 through 2011 • We limited analyses to three classes • Class of 2011: Short-term outcomes in Fall following graduation; opportunity to collect data for this study (SSV) • Class of 2009: Mid-range outcomes 2.5 years following graduation • Class of 2007: Longer-term outcomes 4.5 years following graduation • 68,000 graduates • 16,000 senior surveys • 44,000 Student Tracker records

13. Representativeness of Sample

14. Findings • Just a few sample findings • Many more analyses in report

15. Findings from Early Assessment Program (EAP) • Compared CAHSEE performance to EAP performance • EAP has more rigorous standards than CAHSEE • ELA : Half of CAHSEE Advanced students were EAP Exempt • Math: One-quarter of CAHSEE Advanced students were EAP exempt; two-thirds were conditionally exempt

16. Findings from Senior Surveys • Compared CAHSEE performance to future intentions

17. Post High School Plans

18. Post High School Plans by CAHSEE ELA Achievement

19. Planned Education Level by CAHSEE ELA Achievement

20. Immediate Plans by Socioeconomic Status

21. Findings from Student Tracker • Compared CAHSEE performance to actual postsecondary outcomes • We assumed students not found in Student Tracker followed another path (e.g., military, civilian work)

22. First Enrollment Date by Graduating Class

23. Enrollment Timing by Graduating Class

24. An Important Finding CAHSEE ELA and Mathematics performance are closely related to postsecondary enrollment rates.

25. Lessons Learned • LEA recruitment is time-consuming and labor-intensive. • Clear specifications of expectations are important to facilitate full participation by the LEAs. • Allowing dedicated time for discussion of the study was paramount to its success. • Planning Workshop • Preliminary Results Workshop • Some senior survey items were of limited value and might benefit from revision if a similar study were conducted in the future.

26. Answers to Research Questions 1. What post high school outcomes can be linked to CAHSEE performance? • Links between CAHSEE performance and postsecondary academic pursuits • Links between CAHSEE performance and future intentions of high school seniors • These intentions show some promise for accurately predicting behavior. 2. How well and in what ways does CAHSEE predict post high school performance? • Evidence that CAHSEE performance predicts near-term postsecondary academic pursuits • Weaker evidence that seniors planning to work or join the military may well have done so, based on the absence of evidence that these students pursued higher education • Robust relationship between the10 levels of CAHSEE achievement constructed for this study to postsecondary enrollment 3. How feasible is a collaborative effort among volunteer LEAs to analyze the relationships between CAHSEE performance and post high school outcomes? • Very feasible approach to analyzing these sorts of research questions. • Some LEAs now plan to administer senior surveys on a routine basis. • CDE should consider providing a uniform questionnaire for LEA consideration.

27. A Heartfelt Thanks HumRRO wishes to thank the representatives from the California LEAs who participated in the PHO Study—the individuals who attended workshops, the data analysts who prepared data, and the leaders who granted time and effort to contribute to this study. We appreciate their sharing ideas during the Planning Workshop, their sustaining the effort to provide data—in some cases, implementing new data collection processes such as senior surveys—and their working with us in the Preliminary Results Workshop to interpret and debug findings. Without their commitment and involvement this study would, quite literally, not be possible.

28. Full Report http://www.cde.ca.gov/ta/tg/hs/documents/cahsee2012evalrpt.pdf

29. Contact Information Sunny Becker HumRRO (443) 944-8226 sbecker@humrro.org