Oklahoma’s Experience: Proposed Measure Construction for 2P1-Completion

1. Oklahoma’s Experience: Proposed Measure Construction for 2P1-Completion Phoenix Data Quality Institute June 14-16, 2006 Kathy Spengler Oklahoma State Regents for Higher Education Sarah Mussett Oklahoma Department of Career and Technology Education

2. Background There are two systems of public, postsecondary education in Oklahoma. • Oklahoma State System of Higher Education includes 13 colleges, 2 technical branches, and 12 universities. • The two-year colleges offer associate in applied science degrees, certificates of less than 1 year, and certificates between 1 and 2 years. • Three four-year universities also offer AAS degrees. • Career Technology System includes 29 centers • Technology Centers offer certificate programs to adults at the postsecondary level.

3. Measurement Approach PROPOSED EXIT GROUP: Denominator:2003-04 students in an AAS or certificate program who (1) have completed 1/3 of the credits required in their program and (2) did not enroll the following year (2004-05) Numerator:Students who were awarded a degree or certificate [or who transferred to a 2-year or 4-year institution] CURRENT Denominator: Entry Cohort (1999-00) of first-time entering students who declare an AAS program as their major Numerator:Students who are awarded a degree or certificate within six years

4. Measurement Approach SYSTEMWIDE APPROACH • Identify Concentrators. If the student’s educational goal (theoretically consistent with the major) = AAS or 2-year certificate level, then determine whether the cumulative credit hours >= 20. If the educational goal = 1-year certificate, then determine whether the student had earned at least 10 credit hours. • Identify “Exiters.” If the student was a concentrator in 2003-04, then check for enrollment in any semester of 2004-05. If no enrollment is found in that year, then flag as a “concentrator/exiter” and include in the denominator.

5. Measurement Approach • Identify Degree or Certificate Recipients. Search for degrees awarded in 2003-04 or 2004-05 to match with those students previously identified as “concentrator/exiters.” We used a two-year window to catch many of the “degree-eligible” students who may not have actually received their degrees at program completion (due to outstanding tuition/fines or late applications, etc.)

6. Findings

7. Findings • Overall, a change to the proposed exit group method would result in a higher Oklahoma completion rate than was reported in last year’s CAR: • Completion rates, even including certificate programs, would be almost 4 percentage points higher. • Comparing only AAS students (ignoring students in certificate programs) under the current and proposed completion measure constructions: • More students, in total, would be included under the proposed method. The number of whites and females, in particular, would increase. • Including certificates in the measure would: • improve the completion rates of minority students, up to 5.7% over what would be reported for AAS students only under the proposed plan.

8. Observations, Questions & Issues • Concentrator, eligible coursework – • “Academic and technical coursework that is part of a state established or state approved, locally established CTE program of study may count toward calculating student concentrator status.” • Is it necessary to determine if the cumulative credit hours taken are truly “part of” the CTE program, as opposed to credits which may have been earned during previous study in a different program or may be extra electives that won’t count toward the CTE degree?

9. Observations, Questions & Issues • Concentrator, program sequence – • The proposed definition states that “generally” the average number of credit hours for an AAS degree, two-year certificate, and one-year certificate would be 60 hours, 60 hours, and 30 hours, respectively. • Would there be an expectation that states apply program-specific requirements or institution-specific averages when determining how many hours are needed to be deemed a concentrator?

10. Observations, Questions & Issues • Concentrator, student intent – • Is the use of educational goal to identify students enrolled in AAS or certificate programs appropriate, or is it contrary to the instructions to NOT use student intent in defining a concentrator? If it is inconsistent with our proposed measure, how will other states identify students to be included?

11. Observations, Questions & Issues • PROPOSED CONCENTRATOR ENTRY COHORT • Insufficient longitudinal data were available in this trial run to identify a concentrator entry cohort. • How difficult is it to determine the first point in time that a student attains concentrator status and avoid duplication among multiple institutions or semesters being reported?