Success for All: Multiple Measures for College & Career Readiness

1. Success for All: Multiple Measures for College & Career Readiness Presented at the California Department of Education’s Public Schools Accountability Act Advisory Committee Meeting April 15th, 2014

2. The Linked Learning Alliance A statewide coalition dedicated to improving California’s high schools and preparing students for success in college, career, and life. Employers K-12 Community Org. Industry & Trade Assoc. State & Local Agencies Higher Education

3. The Linked Learning Approach • Linked Learning integrates: • rigorous academics • technical training • real-world learning opportunities • student supports

4. College and Career Readiness Most skilled jobs with living wages require training beyond high school. Neither a college nor career technical training may be appropriate for allstudents, but both are relevant to the aspirations of most students. On-the-Job Training High School Career Community College University Apprentice

5. College and career are neither synonymous nor mutually exclusive. Contextual application of academic and technical skills/knowledge. 21st Century skills and ‘soft’ skills CareerReady College Ready Eligibility for gainful, skilled employment or further training. Technical skills and knowledge Workplace behaviors Eligibility for postsecondary studies without remediation. Academic content knowledge Academic behaviors Cognitive strategies

6. State policy has begun to acknowledge this reality. Statewide Implementation Grants Career Technical Funding Career Technical Programs (CPA, SSP, AgTech, CleanTech) $40 Million +$30 Million (Proposed for 2014-15) Local Control & Accountability Planning $2.1 Billion SB 1070 CTE Pathway Program $48 Million +$48 Million (Proposed for 2014-15) $250 Million Common Core Implementation $1.25 Billion Career Pathways Trust Regional Occupation Centers & Programs $384 Million +$52 Million (Proposed for 2014-15)

7. This convergence builds upon several years of legislative activity working to better align career and college preparation.

8. College Readiness: One size does not fit all • In September, the committee heard about several potential indicators: • A-G Completion • Early Assessment Program (EAP) • Completion of Algebra II or above • Variable observation • None were stable or reliable across school sites • Other discussed indicators: • SAT/ACT, AP/IB and dual enrollment • Are not universally available and/or come at a cost • Likely variation in reliability across school sites • A multiple measures approach is the most realistic, equitable, and feasible. • Career readiness is no different.

9. Career Indicators Demonstrate workforce readiness through acquisition of sector specific competencies and 21st century skills. Many career-relevant indicators are reported to the state: • Perkins • CALPADS More are gaining momentum in districts through: • AB 790 Linked Learning Pilot Program (IEBC) • Career Pathways Trust (IEBC and Cal-PASS+) • CORE districts (School Quality Improvement System) • Local Control and Accountability Plan development

10. Career Indicators Career Pathways Trust • # who have successfully participated in the career pathways program (successfully participated is defined by a “C” grade or better in all pathway courses). • # participating in internships, work-based learning, mentoring, job shadowing opportunities, work experience, and student leadership organizations as part of the career pathways program. • # who received a high school diploma. • # who received a nationally recognized or state approved career technical education (CTE) certificate.

11. Local Control Accountability Plan Career technical education sequences or programs of study that align with state board-approved career technical educational standards and frameworks, including: • Aspirational ROC/ROP Each occupational sequence shall do all of the following: • Result in an occupational skill certificate developed in cooperation with the appropriate employer advisory board. • (2) Provide prerequisite courses that are needed to enter apprenticeship or postsecondary vocational certificate or degree programs. Where possible, sequenced courses shall be linked to certificate and degree programs in the region. • (3) Focus on occupations requiring comprehensive skills leading to high entry-level wages or the possibility of significant wage increases after a few years on the job, or both. • (4) Offer as many courses as possible that have been approved by the University of California as courses meeting the "A-G" admissions requirements.

12. Local Control Accountability Plan • California Partnership Academies Career technical education courses offered at each grade level at the academy that are part of an occupational course sequence that targets comprehensive skills and that does the following: • Provides career technical education in high skill occupations of regional and local economic need. • Focuses on occupations requiring comprehensive skills leading to higher than entry-level wages, or the possibility of significant wage increases after a few years on the job, or both. • Provides a sequence of courses that build upon each other in knowledge, skill development, and experience, and ends in a capstone course that includes an internship component. • Prepares pupils for employment and postsecondary education. Sequenced courses shall be linked to certificate and degree programs in the region, where possible. • Whenever possible, prepares pupils for industry-recognized certifications. • Whenever possible and appropriate, offers career technical education courses that also meet the subject requirements for admission to the California State University and the University of California.

13. Local Control Accountability Plan • Linked Learning • A multiyear, comprehensive high school program of integrated academic and technical study that is organized around a broad theme, interest area, or industry sector, including, but not necessarily limited to, the industry sectors identified in the model standards adopted by the state board. • A program that ensures that all pupils have curriculum choices that will prepare them for career entry and a full range of postsecondary options, including two- and four-year college, apprenticeship, and formal employment training.

14. Local Control Accountability Plan • Linked Learning • An integrated core curriculum that meets the eligibility requirements for admission to the University of California and the California State University and is delivered through project-based learning and other engaging instructional strategies that intentionally bring real-world context and relevance to the curriculum where broad themes, interest areas, and career technical education are emphasized. • An integrated technical core of a sequence of at least four related courses, that may reflect career technical education standards-based courses, that provide pupils with career skills, that are aligned to and underscore academic principles, and to the extent possible fulfill the academic core requirements. • A series of work-based learning opportunities that begin with mentoring and job shadowing and evolve into intensive internships, school-based enterprises, or virtual apprenticeships. • Support services, including supplemental instruction in reading and mathematics, that help pupils master the advanced academic and technical content that is necessary for success in college and career.

15. Data Constraints & Opportunities The State Board of Education can only require currently reported data • CALPADS • Perkins But this doesn’t preclude voluntary reporting of additional data. New, non-statedata systems can offer valuable indicators of readiness among participating schools and districts: • IEBC • Cal-PASS+ • School Quality Improvement System (CORE/Gardner Center) the API!!

16. Categorizing Potential Indicators • Course & Program Indicators • Demonstration Indicators • Postsecondary Experience Indicators • Potential Voluntary Indicators

17. Guiding Principles • College and career readiness • Not either/or • Subsidiarity • Aligned with LCFF/LCAP • Encourage innovation to improve practices and programs statewide • Responsiveness to unique local conditions and realities. • Transparency and relevance • Clear and meaningful to students, families, educators, employers, etc. • Emphasis on student performance over student participation. • Align with, and exceed, readiness standards in other states.

18. Course & Program Indicators Reflect satisfactory performance in a specific program of study and the type of knowledge acquired. May reflect minimum eligibility requirements for postsecondary programs (college, on-the-job training, apprenticeships, etc.).

19. ‘College-Ready’ Course & Program Indicators A-G Completion • Indicates preliminary eligibility AP/IB Course Completion • Signal of more rigorous course of study • Research has linked course participation to improved performance in similar courses at college level.

20. Digital Electronics • Introduction to Engineering Design • Computer Integrated Manufacturing CTE Pathway Completion Sequence of 3 or 4 courses within an industry theme • Central component of career academies • Used in 6 states’ accountability systems as a career readiness option Decades of research have shown connection between career academy completion and improved student outcomes: • Graduation and postsecondary enrollment • Earnings • Credit completion and ‘a-g’ course completion Could meet basic Local Control & Accountability Plan requirement. Sample Engineering & Design Course Sequence

21. Digital Electronics • Circuitry • Simulation design & testing • Electronics troubleshooting • Introduction to Engineering Design • 3D Modelling • Technical Documentation CTE Course Completion (within a pathway) Whereas the a-g course sequence is largely ‘all or nothing’ for UC eligibility purposes,discrete CTE courses aligned in a pathway sequence reflect incremental career preparation via technical skill development. Course 1 Course 2

22. Distribution of points on the index (conceptual example for illustrative purposes) In this scheme, an algorithm could average college and career indicators, with calibration as necessary. & College Career CTE Pathway 800 A-G A-G -2 2 CTE courses 600 + 800 600 = 700 points CTE course 400 2 None of the above None of the above 200

23. Demonstration Indicators Completion of activities that establish a student’s level of experience or competence. • May meet minimum eligibility requirements for postsecondary training programs or employment. College • EAP Ready or EAP Conditional Career • Work-based learning experiences

24. Work-Based Learning Experiences Apprenticeships • Standards-based, industry recognized, and often prerequisite for skilled jobs • Includes clear sequence of experiences aligned to curriculum • Supervisor assessment of readiness WorkAbility • Aligned to Individual Education Plan (IEP) for students with disabilities • Supervisors assessment of job readiness

25. Work Experience Education • Inconsistent standards and quality • Not all are curriculum-aligned or include employer/supervisor assessment • Could be strengthened in statute and regulations Only a handful of states currently use work-based learning as an option for career readiness in their accountability systems. LCAP Implications (when standards-aligned): • Could supplement college- and career-ready baseline • Could be an additional pupil outcome reported in the plan • Career-relevance and connection to curriculum reinforce student engagement and content mastery. • Contextualized learning can reinforce English language acquisition

26. Industry Certification Demonstrates technical competency • Employer recognized, but not applicable in all sectors or industries Schools report acquisition of certifications, but no official list. CDE has been surveying districts to learn which certifications are reported • Ex. Certified Organic Farmer, Veterinary Technician, Certified Nurse Assistant, Microsoft Certified, Collision Repair & Refinishing Requires creation of authoritative list • Ensures level of quality and rigor • Common practice in other states LCAP Implications: • Could be additional pupil outcome included in reporting.

27. Raising the bar on readiness. & College Career CTEP + Ind. Cert or WBL 1000 A-G + EAP Ready WBLEmployer Assessed CTE Pathway 800 A-G A-G or EAP Ready A-G -2 Industry Cert. 2 CTE courses 600 EAP Conditional CTE course CTE course 400 None of the above 200 None of the above

28. Postsecondary Experience Indicators Success in college-level coursesand/or advanced standing upon college entry • If college-level CTE coursework, also demonstrates career preparation Some version of these types of indicators are used in over 20 states as a reporting option for accountability purposes.

29. Degree and Transfer-eligible Units (DTU) Courses that require or demonstrate college-level math or English. Includes credit earned from IB/AP exams and college coursework completed through dual/concurrent enrollment. Place students at advanced standing upon entry to college Provides early exposure to college setting and/or college-level performance expectations Can lead to increased rates of colleges persistence and completion among all students

30. College-level CTE course completion Emerging research shows promise for CTE dual enrollment and students’ postsecondary outcomes. Demonstrates college and career readiness • Shows both college-level competency and employment training LCAP Implications: • Could supplement college- and career-ready baseline • Could be an additional pupil outcome reported in the plan • Rigor and career-relevance can reinforce student engagement, school climate perceptions, and content mastery. • Contextualized learning can reinforce English language acquisition.

31. Diversifying reporting optionsIn this scheme, some indicators could combine college and career, reducing or eliminating the need to average and calibrate. College & Career & College Career WBL + 1 College CTE (Math or ELA pre-req.) A-G + 3 DTU CTEP + Ind. Cert or WBL 1000 30 DTU A-G + EAP Ready WBL + 1 College CTE WBLEmployer Assessed CTE Pathway 800 12 DTU 6 DTU A-G or EAP Ready 1 College CTE Course Industry Cert. 2 CTE courses 600 EAP Conditional CTE course CTE course 400 3 DTU 200 None of the above

32. Diversifying reporting optionsIn this scheme, some indicators could combine college and career, reducing or eliminating the need to average and calibrate. College & Career & College Career A-G + 3 DTU CTEP + Ind. Cert or WBL 1000 30 DTU A-G + EAP Ready WBL + 1 College CTE WBLEmployer Assessed CTE Pathway + 800 800 1000 12 DTU = 900 pts 6 DTU A-G or EAP Ready = 800 pts. Industry Cert. 2 CTE courses 600 2 EAP Conditional CTE course 400 3 DTU 200 None of the above

33. PotentialVoluntary Indicators Supports local control and accountability, through recognition of alternative, indicators or measures. • Incentivizes innovation – drives state towards better measures • A number of promising indicators are already emerging • Alternative reporting and accountability systems could be integrated What would be required? • Evidence of effectiveness • Development of evaluation criteria and review process to obtain PSAA/SBE approval, as well as determine point values.

34. National Career Readiness Certificate (NCRC) Can be earned through successful completion of 3 WorkKeys exams (offered by ACT): • Applied Mathematics • Locating Information • Reading for Information Jobs are profiled to determine the proper skill levels necessary for successful execution of a job or internship. Four skill levels are possible: • Bronze 35% of jobs profiled • Silver 65% • Gold 90% • Platinum 100% Used as an option for career readiness in 13 state accountability systems.

35. Integrated Pathway Completion College and Career Pathway • A-G and CTE • At least two CTE courses meet A-G requirements Increases access to college-ready curriculum for those traditionally tracked into career tech programs. Increases job readiness and soft/tech skill development for college-bound students. LCAP Implications: • Clearly articulates and integrates college and career ready ideals • Career-relevance and connection to curriculum reinforce student engagement and content mastery, improving academic performance outcomes.

36. National Academy Foundation Certification Student-level certification used in almost 100 programs in California. Includes satisfactory completion of three standardized elements: • End-of-course exams in four pathway courses • Four project-based assessments • Supervisor-assessed internship • Could potentially function as an independent indicator due to employer validation, standardization of assessment, and alignment to academic content standards.

37. Completion of Certified Linked Learning Pathway Includes completion of an college and career ready curriculum sequence, in addition to a work-based learning experience. Speaks to the rigor and relevance of a program, as well as the curricular integration across academic, technical and workplace learning. Encourages teaching practices that are effective and evidence-based, as well as equity of access and continuous program improvement.

38. School Quality Improvement System (SQIS) School accountability system for districts participating in the ESEA waiver through the California Office to Reform Education (CORE). Data-driven accountability and continuous improvement system: • Includes academic, socio-emotional and culture/climate indicators • Indicators are evidence-based in their importance for academic achievement and life success (ex. employment, wages, avoidance of risky behavior) Data aggregation performed by third-party. • Gardner Center at Stanford University Schools must meet one of the following benchmarks: • 90% on SQI Index; or • Improvement along index: 2% in 2 years, 4% in 4 years

39. Data Reporting for the Career Pathways Trust Program Enrollment Dual Enrollment Employment Program CompletionInternships Work-Based LearningMentoring Job ShadowingAssociate Degrees Pathway-enrolled Graduates Work ExperienceIndustry Certification Student Leadership Organizations Training Program Entry 4-year Transfer

40. Career Pathways Trust: Data Reporting Systems Individual reporting elements, or aggregates thereof, could be drawn from one of the two reporting systems to meet college and career-readiness accountability requirements. Institute for Evidence-Based Change • Nine districts currently report to this system. • 58 more LEAs/COEs will utilize as participants in the AB 790 Linked Learning Pilot Program Cal-PASS+ • Many LEAs already provide information to this longitudinal system

41. Diversifying options with alternative data reporting. College & Career Career & College WBL + 1 College CTE (Math or ELA pre-req.) Linked Learning Pathway IEBC/ Cal-PASS+ Indicator Integrated Pathway + WBL CTEP + Ind. Cert or WBL A-G + 3 DTU NAF Certified 1000 30 DTU A-G + EAP Ready LL Pathway - WBL WBL + 1 College CTE IEBC/ Cal-PASS+ Indicator WBLEmployer Assessed 90% SQIS (CORE) CTE Pathway Integrated Pathway NCRC Platinum 12 DTU 800 A-G or EAP Ready 6 DTU NCRC Gold 1 College CTE Course Industry Cert. 2 CTE courses 600 EAP Conditional CTE course NCRC Silver 400 3 DTU 200 None of the above

42. Additional Considerations Bonus points to incentivize equitable opportunities and outcomes? • English language learners • Students with special needs • Socio-economically disadvantaged students Algorithm for blending C&CR indicators • Average of each or would additional algebra be required? Calibration of voluntary measures • How might other data systems be validated/integrated? • Cal-PASS+, IEBC, School Quality Improvement System (CORE) • Review process and evaluative criteria

43. Preparing Students for College, Career, and Life All students should have: • Access to an education that prepares them for college and career. • A choice of paths, not a single track. • The ability and support to make informed choices about the future. The Academic Performance Index: • Should recognize the many paths that students may pursue. • Can incentivize schools to offer educational opportunities that provide students with options after high school.

44. Questions? Contact us: policy@linkedlearning.org (916) 248-4848 Visit us: www.linkedlearning.org