Michigan Data Quality Institute January 2006

1. Michigan Data Quality InstituteJanuary 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota

2. Today’s agenda • What tech prep was ‘sposed to be • What the feds think about tech prep • Let’s talk data • What other states are doing. 

3. According to Title II (Section 202), a Tech-Prep program is a program of study that • (A) combines at a minimum two years of secondary education with a minimum of two years of postsecondary education • (B) integratesacademic, and vocational and technical, instruction, and utilizes work-based and worksite learning where appropriate and available; • (C) provides technical preparation in a career field • (D) builds student competence in mathematics, science, reading, writing, communications, economics, and workplace skills through applied contextual academics, and integrated instruction, in a coherent sequence of courses; • (E) leads to an associate or baccalaureate degree or a postsecondary certificate in a specific career field; • (F) leads to placement in appropriate employment or to further education.

4. Elements of a Tech Prep Program Today(USDE) Perkins requires that Tech Prep programs have seven elements: • an articulation agreement between secondary and postsecondary consortium participants; • a 2+2 , 3+2 or a 4+2 design with a common core of proficiency in math, science, communication, and technology; • a specifically developed Tech Prep curriculum; • joint in-service training of secondary and postsecondary teachers to implement the Tech Prep curriculum effectively; • training of counselors to recruit students and to ensure program completion and appropriate employment; • equal access of special populations to the full range of Tech Prep programs; • preparatory services such as recruitment, career and personal counseling, and occupational assessment.

5. National Data on Tech Prep • Only 47 percent of all high schools in 2000 reported offering something they call “Tech-Prep.” • The set of activities pointed to as evidence of Tech-Prep are quite modest in many schools, and some preceded federal support for Tech-Prep. • A highly structured form of Tech-Prep is rarely found in U.S.high schools (10%) • Tech-Prep and non-Tech-Prep students attend college at roughly comparable rates.

6. NAVE: The good news? • Tech-Prep funds have helped spur community colleges to work with local secondary schools on a variety of issues including student recruitment and articulation agreements

7. NAVE: The bad news • Reported participation in Tech-Prep - student increases in student counts - should be viewed cautiously. • Access to and funding of Tech-Prep do not reflect the Perkins Act’s targeting criteria. • Tech-Prep is rarely implemented as a comprehensive program of study; implementation focuses on individual components of Tech-Prep, some of which are becoming more common in vocational education in general.

8. Defining the Tech Prep Student • Student takes/completes articulated vocational courses. 35% • Student develops an individual student plan (indicating a • planned course sequence across the HS & PS lev 27% • Student explicitly elects Tech-Prep as a path, major, • track, or program (e.g.,applies/chooses to be in Tech-Prep) 24% Student takes/completes one or more vocational courses. 19% • Student takes College-Prep-level academic courses. 18% • Student takes/completes one or more applied academic • courses (e.g., principles of technology). 11% • Student participates in work/training experience(s) • at an employer worksite related to a Tech-Prep course. 8% All vocational students are “in Tech-Prep.” 4% A student enrolled in a state-approved secondary career and technical education program that is part of an articulated Tech Prep Program that leads to an associate degree or certificate in a specific career area. Tech-Prep participation is most often defined by enrollment in an articulated course, an increasingly common feature of vocational education in general.

9. NAVE Findings on Tech Prep Elements • Articulation agreements • Work based learning • Applied academics

10. Articulation Agreements: The cornerstone of Tech Prep • Course-to-course articulation rather than on program articulation (Not the vision in legislation) • Few Tech-Prep students receive articulated college credit. • However, articulation efforts have (1) stimulated communication between secondary and postsecondary vocational faculty and (2) improved the rigor and consistency of some secondary vocational curricula, by encouraging high schools to adopt college curricula and instructional materials as proof of course equivalency

11. Work Based Learning • Worksite experiences are not a focus of recent Tech-Prep efforts. • Most of the workplace experiences in which Tech-Prep students engaged were available to all students in their schools rather than targeted specifically to Tech-Prep students.

12. Applied Academics • Early enthusiasm has waned somewhat because: • “Off-the-shelf” applied curriculum packages were expensive • Stigma problem • Tech-Prep efforts to encourage academic teachers to develop and adopt applied approaches has not been widespread • Pressures of state academic standards and high school exit exams have made it more difficult to promote these approaches

13. No Progress in PS Settings • Tech-Prep is largely viewed as an effort to upgrade secondary vocational education; it has little impact on postsecondary courses. • Tech-Prep has little impact on college curricula and services. • Relatively few postsecondary institutions have offered new courses for students entering with Tech-Prep experience or modified other courses • No data are available about the proportion of students who complete a two-plus two program or the benefits of doing so.

14. NAVE Tech Prep Conclusions • Tech-Prep was a catalyst for certain vocational reform activities but, because few schools implement it as a comprehensive program of study, it is now playing less of a distinctive role • Tech-Prep efforts now overlap substantially with those of regular vocational education. • Eliminate Tech-Prep as a separate title, folding its key activities into postsecondary institutions’ responsibilities.

15. The Big Problem: Your Problem . . . Inadequate data systems limit the tracking of Tech-Prep students into postsecondary partner institutions.

16. In a Perfect World, Tech Prep is a set of Structured Programs • That combine the different elements outlined in the law for those students who choose to participate. • Where students would commit to a program’s career focus • That include a well-defined sequence of academic and vocational courses leading to employment in a specific career field • Where students would be clustered together in many if not most classes. • Where secondary programs would identify the particular program(s) at the postsecondary level in which students are expected to continue, and • Where Articulation agreements really work • Where PS institutions would offer a differentiated credential as a signal to employers that the Tech Prep student was special.

17. Data Who Uses It and Why

18. General Notions about Data Quality • Reliability and validity • Utility • Objectivity • Integrity

19. Reliability • Reliability is the extent to which any measuring procedure yields the same result on repeated trials.

20. Validity • Validity refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure. • External validity refers to the extent to which the results of a study are generalizable • Internal validity refers to the rigor with which the study was conducted

21. Data Reporting—US DOE Guidelines • Utility: • Usefulness of the information to its intended users. • It is achieved by staying informed of information needs and developing new products and services where appropriate. • Care must be taken in the review stage to ensure that the information can be clearly understood (and reproduction).

22. Data Reporting—US DOE Guidelines (Ctd) • Objectivity: • Accuracy, reliability, and unbiased nature of information. • Itis achieved by using reliable information sources and appropriate techniques to prepare information products. • Objectivity involves both the content and the presentation of the information.

23. Data Reporting—US DOE Guidelines (Ctd) • Integrity: • Security or protection of information from unauthorized access or revision. • Integrity ensures that the information is not compromised through corruption or falsification. • Statutory and administrative guidelines to protect the integrity of Department information include the following: • Privacy Act; • Freedom of Information Act; • OMB Circulars A-123, A-127, and A-130; • Federal Policy for the Protection of Human Subjects; • Family Educational Rights and Privacy Act; • Computer Security Act of 1987; • Government Information Security Reform Act; and • National Education Statistics Act, as amended by the USA Patriot Act.

24. Perkins—Tech Prep data: Accountability 3S1 • Performance indicator Placement in, retention in, and completion of, postsecondary education or advanced training, placement in military service, or placement or retention in employment.

25. Tracking the Tech Preppie: Possible Trajectories Stop Out Re- medial 11th Grade + 12th Grade + PS Year 1 + PS Year 2 Non Related PS Drop Out Drop Out Out of State

26. Student Trajectories

27. What are other states doing for Tech Prep Data Collection (7 States) • States with a full data collection process (secondary and postsecondary cooperation): NY (and FL) • States with data collection process carried out at the college level: OH, NC, CO, CA, CT. • Colleges use different methods (e.g., record matching) • Some states survey the college

28. Tech Prep Data Collection Process Key Elements • Definition of tech prep program (or student) • Express commitment • Clarify the role of the designated officer • Identification of Tech Prep Student • Measure Progress • HS completion/program completion • Transition to: 2-year college, 4-year college, job, apprenticeship, military, other • Postsecondary Enrollment • Progress • Completion • Post completion: work in related/unrelated area, other 2-year college degree, 4-year college, military, or other

29. Tech Prep Data Collection—DefinitionMichigan • Definition of tech prep/tech prep student • A student enrolled in a state-approved secondary career and technical education program that is part of an articulated Tech Prep Program that leads to an associate degree or certificate in a specific career area.

30. Tech Prep Data Collection—Full Model Practices: NY (and FL) • Contract or agreement • Contract signed by student and parent(s) (NY) • Contract signed only if participating school & school district agrees to it (CO) • Represents the first step towards identification/progress track (record is flagged) • Allows for: • Use of student identification (SSN,UI, Name) • Early data entry (NY & FL) • Early intervention of officer in charge (NY) or information system (flag, in FL)

31. Tech Prep Data Collection—Full Model Practices: NY (and FL) • Role of designated officer • Track progress • Secondary (Meeting standards?) • Postsecondary • Out of education system • Ensures data collection process • School official (secondary) (NY) • Tech prep director (postsecondary) (NY, also CO) • High school officials / School district officials (FL)

32. Tech Prep Data Collection—Full Model Practices: NY (and FL) • Student identification • Flag (in transcripts or in the information system, in FL) • Based on contracts (NY) • [SSN, first/last name, birth date (CO)]

33. Tech Prep Data Collection—Full Model Practices: NY • Progress (NY only) • Secondary level (school official): • Meet tech prep/high school standards • Grades and courses • HS completion

34. Tech Prep Data Collection—Full Model Practices: NY (and FL) • Postsecondary enrollment • Based on school or school district reporting at secondary level (NY). Report sent to PS. • Record matching once enrolled in the postsecondary level, since information is shared and student record has been flagged (FL)

35. Tech Prep Data Collection—Full Model Practices: NY (and FL)-G 7. Progress (NY only) Postsecondary level (tech prep director): • Next steps after HS graduation: work, the military, apprenticeship, college (also in FL) • If community college: • Student progress • Graduation • Degree obtained • Within six months of graduation: Work (related/unrelated field), further education, military

36. Computing the 3S1 Ratio in MI Numerator: The number of 12th grade program completers who graduated the previous year and are in postsecondary education or advanced training, employment, and/or military service. Denominator: The number of 12th grade program completers who graduated from school the previous year. State Developed and Locally Administered Survey

37. Summary of options for improving data quality • Incorporate SSN in K-12 records • Incorporate K-12 unique identifiers in PS records • Conduct probabillistic matching of HS and PS student records using FERPA available data • Conduct electronic (web based) surveys • Automate mail surveys (if used); combine with web surveys

38. Tech Prep Data Collection—Obstacles • Lack of (or limited) definition for tech prep program and/or tech prep student that adjusts to state needs • No commitment with the program (contracts) • Non complimentary data system

39. Tech Prep Data Collection—Obstacles 4. Identification • Data privacy • Not everybody has the same designator • School districts or families oppose • Alternatives: • Flags (integrated system) • Contract

40. Tech Prep Data Collection—Data Quality Issues • Ensure quality of early data entry (FL) (garbage in; garbage out) Reporting reviews • Matching records: Identifiers (ssn)

41. Tech Prep Data Collection—Additional Issues • Colleges cannot know, from transcripts, whether the student was in a tech prep program at moment of application to college. (HS transcripts are often issued earlier than completion) • Data collection comes from the college perspective, but some information maybe lost if secondary are not reported as well

42. Tech Prep Data Collection—Additional Issues • Data are not collected if student is in general education or is not a full time student at the college level • Data are collected for students that have declared to be in tech prep at the secondary level only if they are enrolled in a community college, regardless of continuing or not in a tech prep program • There is no tracking of high school students in tech prep if they go to another school

43. Questions / comments James R. Stone III National Research Center for Career and Technical Education—UMN 612-624-1795 stone003@umn.edu