Comprehensive & Annual Program Review Data Delivery Plan

1. AY 2007 Comprehensive & Annual Program Review Data Delivery Plan Data Description Process – Timeline Rev. 11-27-07

2. Purpose • The primary purpose of this presentation is to provide clarity to all instructional programs on where and how the data is calculated for both the Comprehensive and our system required Annual Program Reviews. • We have been asked to produce an annual program review for each and every one of our instructional programs and units. They are required of each system CC and will be taken to the U of H Board of Regents for their review. • The system has changed it’s scheduled due date for annual reviews to be nearly on top of our normally scheduled comprehensive reviews. We may want to reevaluate the scheduling next year to see if it makes more sense to spread the reviews out a little more. • So who has to do a program review this year? If you are normally scheduled to do a comprehensive or are “Jumping” you will do a comprehensive review this year. Additionally, every instructional and non-instructional program will do an Annual Program Review this year. • Not sure if you’re scheduled for a comprehensive or not? Click here for the Comprehensive Program-Unit Schedule.

3. What’s a Jumper? • Jumper is a technical term that is used to describe an instructional program or non-instructional program (unit) that has decided to jump out of their normally scheduled slot for their comprehensive program reviews and into this years cycle. • Why would anyone want to do that? Jumping into this years comprehensive cycle means that you will have an opportunity to be considered for any budgetary decisions that will be made in this years biennium budget process. • Then I don’t have to do another comprehensive for another 4 years? Not so fast…Jumpers will still have to do their comprehensive review on their next scheduled review. Jumping does not affect the existing schedule—you are voluntarily doing an extra review to be considered in this budget cycle.

4. I belong to an Instructional Program… which template do I use? • A datasheet will be provided to all instructional programs for use in both the comprehensive and annual reviews. • There are 3 templates available to Instructional Programs this year. Simply click on the template below that applies to your current situation, paste in the completed datasheet where needed, and then fill in the template: Instructional Comprehensive Program Review Template (Use this template ONLY if you are scheduled for a comprehensive program review this year or are jumping) AY 2007 Annual Program Review Template (ALL Instructional programs will need to complete this) AY 2007 Annual Program Review Coversheet (ALL Instructional programs will need to complete this)

5. I belong to a non-instructional unit…What template do I use? If you are on the schedule for a comprehensive unit review or are jumping you will be using the comprehensive unit review template below. Comprehensive Unit Review Template • For annuals, if you are writing your unit review in support of Administrative Services you will use: Administrative Services Template • For annuals, if you are writing your unit review in support of Student Services you will use: Student Services Template • For annuals, if your unit is other than Student Services or Administrative Services you will use: Generic Annual Unit Review Template • For annuals, ALL units need to complete the following coversheet, which is required by our System Office. Annual Unit Coversheet (not available from system yet)

6. Instructional Program Paid Courses • Since we currently have no way of determining or flagging your program courses in our student information system, we have asked for your help in identifying them. • Today’s process is to look at all courses taught over the last full academic year, add all of our new courses to that list, and then work with the Assistant VCAA to help place all courses in the correct program. • Before using the PPC’s in our program review we will do a validation with the Division Chairs to ensure that the courses listed under your program are in fact your program paid courses. This gives us an opportunity to close any gaps prior to running the data, which should help to eliminate any reruns. • In short, the PPC are the courses that your program is responsible for. They are the courses taught by the programs faculty and lecturers who are paid by the program. • This may include courses taught by the programs faculty and lecturers that are paid by the programs grant money as well—they are still being paid by the program. • Not sure what’s listed for your program? Check the Master PPC Listing

7. Program Paid Courses are… • All Liberal Arts A.A. degree courses taken directly from the General Catalog EXCEPT for elective courses. • PPC as defined by DC/Program Initiators, and Assistant Dean’s of Instruction. • Any course described as an Experimental course with a course number such as 197, 198, 297, or 298.

8. Courses Not Counted asProgram Paid • Any course described as Coop Voc Ed such as 93V or 193V. • Any course described as Independent/Directed Studies such as 99V, 199V, or 299V. • Any Liberal Arts A.A. degree elective courses. • Any Liberal Arts A.A. degree non-elective courses under 100 level.

9. AY 2007 Comprehensive-Annual Instructional Program Review Data Elements • At this point in time we do have a working routine to generate your data. It will take a couple more days of data checking before we can deliver your completed data tables. We are hoping to deliver the data tables by Friday, November 2nd…earlier if humanly possible. • The data tables you will be receiving will cover the last 3 full academic years AY04-AY07 and lists 32 different data elements from a variety of sources. • Student information data this year comes exclusively from the Operational Data Store (ODS). Organizationally this means that all community colleges are getting the data from the same place and at the same snapshot in time (this is a good thing). • The following slides will explain in detail what data has been provided to you for your comprehensive-annual instructional program review write ups and how it has been calculated.

10. What’s different this year? • This year the system has provided the routines that go out and get your data. This is a good thing because we can now begin to document exactly how the data is being queried, what student information system is being queried, the table and field names used in the query, and how the data is calculated. • The system is not providing information on program budgets, expenses, program health calls, or PHI data so we will have to collect our own data for the program review. • Since we have no UHCC agreed upon methodology for determining what a program course is, the system developed the routines to only look for program courses with the same subject code. This means that a program like ABRP will run fine because all of the program paid courses (PPC) (our local definition for a program course) have the ABRP subject code. A program that does not have all of the same subject codes listed in it’s PPC will not return all of the course data for the program. For example, using the system routines, a program like Accounting will not return any of the Business courses, Ent 197, or IS 55. • This means we need to write the routines that enable us to get all of our course data, program budgets, expenses, program health calls, and PHI data that the system routine leaves out.

11. A word about terminology • Since we are using the ODS this year to get the majority of the data a short primer on terminology would be prudent. The following data elements are used in the queries we build to get much of your data. • Many of the routines filter on the students in a field called “IRO Institution”. This is the UH institution at which a student is registered for credit classes. • Freeze Event is locked down on either Census or End of Semester (EOS). The freeze event allows all institutions to look at the data at the same snapshot in time. This is a critical element for an organization that desires to consistently report the same values based on when the data was frozen. Census is the 5th Friday after the start of instruction, and EOS is 10 weeks after the end of instruction.

12. #1 Annual new and replacement positions in the state. • This data element represents the combined new and replacement jobs for the State of Hawaii in your trade, projected for the period 2005-2011. • The data comes from the system office in a report created by the EMSI application they have license to. • EMSI stands for Economic Modeling Specialist Inc. • From their website, “…EMSI specializes in reports that analyze and quantify the total economic benefits of community and technical colleges in their region, and also creates data-driven strategic planning tools that help colleges maximize their impact through labor market responsiveness…”

13. #2 Annual new and replacement positions in the County. • This data element represents the combined new and replacement jobs for the County of Hawaii in your trade, projected for the period 2005-2011. • The data comes from the system office in a report created by the EMSI application they have license to. • EMSI stands for Economic Modeling Specialist Inc. • From their website, “…EMSI specializes in reports that analyze and quantify the total economic benefits of community and technical colleges in their region, and also creates data-driven strategic planning tools that help colleges maximize their impact through labor market responsiveness…”

14. #3 Number of Majors • This is total number of students registered at Hawaii Community College for credit classes within the declared major. • This is as of Fall semester at census.

15. #4 Student Semester Hours for program majors in all program classes • This is the sum of semester hours for registered students in credit classes within the declared major. • This is as of Fall semester at census. • Semester hours are defined here as the “Attempted” semester hours—not earned credits. • Note that “program classes” here refers to your program paid courses (PPC’s). • SSH excludes Directed Studies courses (99, 099, 199,299) but includes Cooperative Ed courses (93, 093, 193, 293)

16. #5 Student Semester Hours for non-program majors in all program classes • This is the sum of semester hours for registered students in credit classes not within the declared major. • This is as of Fall semester at census. • Semester hours are defined here as the “Attempted” semester hours—not earned credits. • Note that “program classes” here refers to your program paid courses (PPC’s). • SSH excludes Directed Studies courses (99, 099, 199,299) but includes Cooperative Ed courses (93, 093, 193, 293)

17. #6 Student Semester Hours for all students in program classes • This is the sum of semester hours for registered students in credit classes without regard for the declared major. • This is as of Fall semester at census. • Semester hours are defined here as the “Attempted” semester hours—not earned credits. • Note that “program classes” here refers to your program paid courses (PPC’s). • SSH includes Cooperative Ed courses (93, 093, 193, 293)

18. #7 StudentFTE Program Enrollment • This is the sum of semester hours for all registered students in credit classes without regard for the declared major (from data element #6 ) / 15 credits for full time. • This is as of Fall semester at census. • Semester hours are defined here as the “Attempted” semester hours—not earned credits.

19. #8 Number of Classes Taught • This is the number of program paid courses (actual sections) taught in the program. • This is as of Fall semester at census. • Semester hours are defined here as the “Attempted” semester hours—not earned credits. • The number of classes taught excludes Directed Studies courses (99, 099, 199,299) but includes Cooperative Ed courses (93, 093, 193, 293)

20. #9 Determination of program’s health based on demand (Healthy, Cautionary, or Unhealthy) • This determination has yet to be decided…who makes call, when call gets made, etc.

21. #10 Average Class Size • Average class size is the sum of registrations in each of the PPC classes divided by the total number of classes. • This is as of Fall semester at census. • This excludes Directed Studies courses (99, 099, 199,299) but includes Cooperative Ed courses (93, 093, 193, 293)

22. #11 Class Fill Rate • Class fill rate is the sum of registrations in each of the PPC classes divided by the sum of maximum enrollments. • This is as of Fall semester at census.

23. #12 FTE of BOR Appointed Program Faculty • FTE of BOR Appointed Program Faculty is the sum of appointments to your program (1.0, 0.5, etc) • Remember that these are positions that were appointed to your program—whether or not these people are actually teaching classes or not. • If a professor is on sabbatical for instance they are still included in this number. • This is for the academic year.

24. #13 Student Faculty Ratio • Student faculty ratio is the number of student majors (from data element #3) divided by the number of BOR Appointed Program Faculty (from data element #12) • Student Majors are taken as of Fall semester at census. • Note: this is not all students taking your classes…just the majors.

25. #14 Number of Majors per FTE Faculty • Total number of fall majors (from data element #3) divided by FTE faculty. • FTE faculty is a workload measure not to be confused with BOR Appointments. It is calculated by taking all of the semester hours taught in PPC classes and dividing them by 15 credits for full load. • Student Majors are taken as of Fall semester at census.

26. #15 Program Budget Allocation • Program budget allocation should include: Personnel, supplies and services, and equipment. • We will calculate personnel costs by multiplying the Faculty FTE, which includes lecturers, by the AY 2007 UHPA faculty rank 4 rate per credit hour value of $1551, then by 30 credits. • Personnel costs plus b budget costs equal the program budget allocation. • Perhaps this element should be labeled, “Program Costs” as these are expenses to the program…

27. #16 Cost per Student Semester Hour • This is the cost to run your program based on student semester hours. • Costs come from Program Budget Allocation (data element #15) divided by student semester hours for all students in program classes (from data element #6) • It should be noted that we are dividing the number of student semester hours from the Fall semester by an annual figure.

28. #17 Number of classes that enroll less than 10 students • This is the number of PPC classes where the student registrations are less than 10. • These are registered students in fall semester as of census.

29. #18 Determination of program’s health based on efficiency ( Healthy, Cautionary, or Unhealthy) • To be determined.

30. #19 Persistence of Majors Fall to Spring • Student records for the declared major in the Fall semester at census are compared to the student records in the declared major in the Spring semester. • Spring records that were also in the fall are persisting. • Persistence fall to spring = Spring persistence / Fall majors. • Example: 31 majors start in Fall 21 majors persist into Spring 21/31 = .6774 or 67.74%

31. #20 Number of Degrees Earned (annual) • The number of degrees earned are counted as of the fiscal year. For example, AY 2007 is roughly equivalent to the 2007 fiscal year. • We are counting the student major outcomes such as AAS, AA, AS, etc.

32. #21 Number of Certificates Earned (annual) • The number of certificates earned are counted as of the fiscal year. For example, AY 2007 is roughly equivalent to the 2007 fiscal year. • We are counting the student major outcomes such as CC, CA, etc.

33. #22 Number of Students Transferred to a 4-Year Institution • This is the number of students that were enrolled in the 0607 academic year at UH Manoa, UH West Oahu, or UH Hilo for the first time… • And, not enrolled UHM, UHWO, or UHH at any time in the previous academic year… • And, had an enrollment at HawCC at any time in that same previous academic year… • And their cumulative earned student semester hours were more or equal to 12.

34. #23 Perkins Core Indicator: Academic Attainment (1P1) • Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. The data that you see in your program review is actually representative of the previous academic year. • Academic Attainment is calculated by: Concentrators who have a cumulative GPA >= 2.0 in academic courses and who have stopped program participation in the year reported Concentrators , with academic courses, who have stopped program participation in the year reported

35. #24 Perkins Core Indicator: Technical Skill Attainment (1P2) • Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. The data that you see in your program review is actually representative of the previous academic year. • Technical Skill Attainment is calculated by: Concentrators who have a cumulative GPA >= 2.0 in vocational courses and who have stopped program participation in the year reported Concentrators who have stopped program participation in the year reported

36. #25 Perkins Core Indicator: Completion Rate (2P1) • Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. The data that you see in your program review is actually representative of the previous academic year. • Completion rate is calculated by: Concentrators who have received a vocational degree or certificate and who have stopped program participation in the year reported Concentrators who have stopped program participation in the year reported

37. #26 Perkins Core Indicator: Placement in Employment, Education and Military (3P1) • Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. The data that you see in your program review is actually representative of the previous academic year. • Placement in employment, education, and military is calculated by: Completers in the year reported (previous Perkins year) who have stopped program participation and who transferred or are employed within one employment quarter following program completion Completers in the year reported (previous Perkins year) who have stopped program participation

38. #27 Perkins Core Indicator: Retention in Employment (3P2) • Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. The data that you see in your program review is actually representative of the previous academic year. • Retention in employment is calculated by: Numerator 3P1 retained in employment for a second quarter and continuing transfers Completers in the year reported (previous Perkins year) who have stopped program participation and who transferred or are employed within one employment quarter following program completion (Numerator of Report 3P1)

39. #28 Perkins Core Indicator: Non-Traditional Participation (4P1) • Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. The data that you see in your program review is actually representative of the previous academic year. • Non-Traditional participation is calculated by: Participants in under-represented gender groups who participated in nontraditional programs in the year reported Participants in nontraditional programs in the year reported

40. #29 Perkins Core Indicator: Non-Traditional Completion (4P2) • Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. The data that you see in your program review is actually representative of the previous academic year. • Non-Traditional participation is calculated by: Completers in under-represented gender groups in nontraditional programs in the year reported Completers in nontraditional programs in the year reported

41. #30 Determination of Program’s Health Based upon Effectiveness (Healthy, Cautionary, Unhealthy) • To Be determined…

42. #31 Determination of Program’s Overall Health (Healthy, Cautionary, Unhealthy) • To Be determined…

43. #32 Number of FTE Faculty • FTE Faculty is determined by taking all of the credits that instructors earn by teaching a course and summing them up, and then dividing by 15 credits for the full time equivalent value. • This is Fall semester only. • The courses that are considered in this calculation are your PPC courses for your program. • The number of FTE Faculty includes both Faculty and Lecturers.

44. Comprehensive & Annual Program Review Timeline All instructional program and unit reviews (Comprehensive and Annual) due to VCAA by email by EOB Friday November 30th Validate program/unit listing held by system and return Develop process, training materials, and timeline Delivery of 44 Program & Unit Reviews to System Office Oct 3-26 12Oct07 30Nov07 Dec 14 When Reviews are complete. October 26th Oct 16-26 November 2nd Develop 2007 program review in the assessment website Provide Annual Program Review Training to Campus Deliver data for instructional programs Post 44 Program & Unit Reviews to Assessment Website

45. AY2007 Comprehensive & Annual Review Process • Step 1 Write your instructional or unit review using the appropriate template's. • Step 2 Send yourdocuments to VCAA Doug Dykstra by email no later than end of business, Friday November 30th. • Step 3 VCAA will ensure that all reviews and coversheets have been received and that they are adequate. • Step 4 VCAA will forward all approved reviews to IR Shawn Flood for further processing. • Step 5 The Annual reviews will be appropriately packaged and sent to the System Office for review by the UH Board of Regents. Comprehensive reviews will be forwarded along as appropriate. • Step 6 All reviews will finally be converted to PDF and posted to the Assessment Web Site.

46. Questions? • The intention of this presentation was to provide one source for all of the documentation related to the Comprehensive & Annual Program Review process. I have linked all of the documents you should need directly into this presentation. • If you need more information on this process please feel free to contact me: Shawn Flood974-7512 Mahalo!