2015 ARPD General & Pre-Professional Program Review

1. 2015 ARPD General & Pre-Professional Program Review Data Description ARPD Process Rev. 18Sept15

2. Covered in today’s session… • Quick review of Program and Unit Review website • Navigating the Annual Reports of Program Data (ARPD) website • Schedule • Process • Data Table Organization • Terminology used in the review • Data definitions

3. Programs Covered in this Presentation • The following General and Pre‐Professional Education programs: • Liberal Arts Program • Hawaiian Studies Program • Natural Sciences Program

4. Reason for Program Review • The review of a program should be an on-going, year-round, reflective process.  • Program review processes assure educational quality and help programs to evaluate and improve their services. • Program review is an opportunity for self-study, self-renewal, and an opportunity to identify the need for improvement. • Your program review may be one of the few opportunities you have to showcase the accomplishments of your program. Take this opportunity to shine. • A robust program review process is one mechanism available to the college to improve student success. • Learn more about how our program and unit reviews support institutional effectiveness here Rubric for Evaluating Institutional Effectiveness

5. About the ARPD Web Submission Tool • If you have attended the training session in-person today you can skip the slides that deal with Navigating the ARPD site. I’ve included the slides here for those that may not have been able to join us today for the live demonstration. • The Annual Reports of Program Data Web Submission Tool (or ARPD for short), is a repository for Annual program and unit reviews for: Instruction, the Academic Support Unit, and Student Support Services. Much of the data that you will need to complete your review has been provided by the Office of the Vice President for Community Colleges. • The ARPD is a home-grown tool, developed in-house by the Office of the Vice President for Community Colleges (OVPCC), and was developed specifically to meet the needs of the community college system.

6. ARPD Web Submission Tool Best Practices • Your programs data table is available on-line within the toolby August 15th every year. You should have everything you need to begin writing your review. • Plan to save your work often—especially when switching between screens, and plan to do most of your formatting within the tool if you are copying and pasting in from Word. Also plan to spell check and save your review in Word before starting. • One of the nicest features about the ARPD is that you can go back and look at reviews from previous years. You can also enhance your own review by leveraging off similar reviews at other institutions.

7. Navigating the ARPD Web Submission Tool UHCC Annual Report of Program Data Web Submission Tool • Begin by clicking on the link above. To peruse the site select any review year, college, or program. To enter information on your review, click on the button on left of the screen called, “2015 Instructional”, then click on the “Web Submission” button. You will be asked to log in by typing in your UH username and password, in order to get to the web submission site. The default screen will take you to the “Status” tab, where you will be able to view whoever was the last user to modify the information in your review. • You can enter the information about your program in any order you wish, but moving from left to right across the tabs at the top of the screen follows the same logical path we used when we were filling out the templates in the past. Start by clicking on the “Analysis” tab. • On the Analysis screen you can either go in and just preview what has already been inputted by clicking on the “Preview” button, or you can go to the edit screen and begin entering or updating information. Select your program.

8. Analysis Tab • On the Analysis screen begin editing by going to the bottom of the page, under the data sheet. Click the edit button and scroll down. There are 3 sections for you to edit here: Analysis of your Program, Action Plan, and Resource Implications. Simply click on the “Edit” link and you will be taken to a screen very similar to MS Word, where you can begin typing in your analysis. (Note the Save button in the top left hand corner!) • In the Analysis of the Program section, based on your ARPD data you will need to: analyze the Program in terms of Demand, Efficiency, and Effectiveness. Include significant Program actions (e.g., new certificates, stop out, gain/loss of positions) and results of prior year's action plan. Include analysis of any Perkin's Core Indicator(s) for which the Program’s goal was not met. Also discuss any trends or other factors (internal/external) affecting the Program and analyze other Program changes or information not included elsewhere. • In the Action Plan section you will describe in detail the Program’s overall action plan for the current/next academic year. Discuss how these actions support the College's Mission and can lead to improvement(s) in student learning. Include specific action plans to address any ARPD Health Call scores of “Cautionary” or “Unhealthy,” and any Perkin's Core Indicator(s) for which the Program’s Goal was not met.

9. Analysis tab continued… • If you are requesting additional people, services, or equipment for your program, you will need to make the justification in the “Resource Implications” section. For this section please provide a brief statement about any implications of current operating resources for the Program. • Budget asks are included in the 3-year Comprehensive Review, except for the following that may be included here: health and safety needs, emergency needs, and/or necessary needs to become compliant with Federal/State laws/regulations. Describe the needed item(s) in detail, including cost(s) and timeline(s). Explain how the item(s) aligns with one or more of the Strategic Initiatives of the Hawai'i Community College 2015-2021 Strategic Plan. Identify and discuss how the item(s) aligns with the Initiative’s Goal, Action Strategy, and Tactic

10. Description tab • Click on the “Description tab” and provide a short summary regarding the last comprehensive review for this program. Discuss any significant changes to the Program since the last comprehensive review that are not discussed elsewhere. • Also, provide the URL to the specific location of this Unit’s last Comprehensive Review the archives at the HawCC Program/Unit Review website • You should be able to copy and paste the link right off the program-unit review website for the year of your last review. • Click on the “P-SLOs tab” and go to the next slide…

11. Program Student Learning Outcomes PLOs = P-SLOs For all parts of this section, please provide information based on the PLOs (P-SLOs) that were assessed through PLO-aligned course assessments in AY 2014-15. • Evidence of Industry Validation • General Pre-Professional Programs can skip industry validation. • Primarily for CTE Programs: • Provide documentation that the program has submitted evidence and achieved certification or accreditation from an organization granting certification in an industry or profession.  If the program/degree/certificate does not have a certifying body, you may submit evidence of the program’s advisory committee’s/board’s recommendations for, approval of, and/or participation in assessment(s).

12. B. Expected Level of Achievement For each Course assessed in AY 2014-15: Discuss the rubric(s) standards and the benchmark goal(s) for student success (e.g., “85% of students will achieve Excellent or Good ratings in the assessed activity” or “90% of students will score Meets or Exceeds Standards on the assessment rubric”).

13. C) Courses Assessed List all Program Courses assessed during AY 2014-15. Also list Program Courses for which a follow-up “Closing the Loop” assessment was implemented in AY 2014-15.

14. Assessment Strategy/Instrument  • For each Course assessed in AY 2014-15, provide a brief description of the assessment strategy, including: • the type of student work or activity assessed, • how and when the assessment was conducted, • how and why assessed artefacts were selected, and • how the artefacts were analyzed.

15. Results of Program Assessment • For each Course assessed in AY 2014-15: • provide a summative description of the assessment results, • discuss how these results collectively demonstrate achievement of the Program’s Learning Outcomes, and • discuss how these results support the College’s Mission.

16. Other Comments • Include any additional information that will help clarify the assessment results. Include comparisons to any applicable College or Program standards, or to any national standards from industry, professional organizations, or accrediting associations. Include, if relevant, a summary of student survey results, CCSSE, e-CAFE, graduate-leaver surveys, special studies, or other assessment instruments used. • Next Steps • Based on the Program’s overall AY 2014-15 assessment results, describe the Program’s intended next steps to enhance instruction in order to improve student learning.  Instructional changes may include, for example, revision to curriculum, teaching methods, learning outcome statements, student support, and other options. Please note here if proposed changes will involve Program and/or Course modifications requiring approval.

17. “Cost Per SSH” tab in ARPD • There are 4 different values that need to be added on the Cost Per SSH screen in ARPD. Basically, all of the fund amounts that will be entered are used in the calculation for the cost per SSH for your program. The costs that need to be entered are: • General Funds • Federal Funds • Other Funds • Tuition and Fees • The definitions for what comprises each of these funds, are laid out in detail in the data definitions section of this presentation, so will not be duplicated here. • Joni will be uploading all of the budget information you need this year. Once the values are loaded and saved in the tool, the values for “Total Funds” and “Cost per SSH” will auto-calculate into your datasheet.

18. “External” tab in ARPD • The “External” screen is intended for programs that utilize external licensures. Currently, the only program at HawCC with external licensures is the Nursing program. • If you are not in the Nursing program you can skip this tab/screen altogether because the first question is, “Does this program utilize external licensures?” The radio button defaults to “No”. • If you are in the Nursing program please click the “Yes” radio button to answer the question and then enter the Number sitting for exam, and the Number passed. Do this for each program where you utilize external licensures. The percent passed will auto-calculate into your datasheet when you click the “Save External Data” button. • The Vice President of Community College’s office collects this data as part of our annual Graduate-Leaver reporting as well as the UH system, “Measuring our Progress” report.

19. “Capacity” tab in ARPD • The “Capacity” screen is only intended for programs that have an externally mandated capacity. • The following criteria is used as an alternate measure for the Student/Faculty Ratio measure within the Efficiency Health call: • “If your program has an externally mandated (e.g. professional accreditation or licensing) capacity of less than 16 students per faculty, the program may be eligible for the alternative efficiency health call calculation.” • If your program fits the criteria listed above, AND your Efficiency Health call is other than Healthy, AND your efficiency health call can be improved by using the alternate method… Please contact: Cheryl Chappell-LongDirector Academic Planning, Assessment, and Policy AnalysisPhone: 808-956-4561Email: cchappel@hawaii.edu

20. “Users” tab in ARPD • The users tab will provide a list of individuals have been identified by the College Vice Chancellor of Academic Affairs as having a specific program role in the analysis of Annual Report of Program Data. • If you need to make any changes to this list, contact the VCAA.

21. “Help” tab in ARPD • The “Help” screen is a very useful resource for some of the documentation needed to support your program within the review process. It contains all of the glossaries and health call scoring rubrics for each year we have used the online tool. • Be sure to use the corresponding glossary and health call scoring rubric for the review year you are interested in.

22. “Program Courses” tab in ARPD • The list of courses that are core to your program are listed here. Many of the data elements use this list to calculate values within your review. Over the course of the Summer every year, the VCAA’s on each campus are asked to submit the current listing of program courses, so that they can be programmed into the upcoming program review. • Take a moment to ensure that the courses listed here are correct for your program and be sure to report any discrepancies to the VCAA, so that they can be corrected.

23. Do I have to write a review this year? • All programs and units are required to submit an annual review this year. If you are listed on the schedule linked below you will be submitting an annual. • If your program is due for a comprehensive review, then you will additionally have to submit a comprehensive review. (Note: this is a different process from last year’s) 3-Year Comprehensive Program Review Schedule

24. Process for Completing the ARPD Instructional Program Review ANNUAL REVIEWS All General Pre-Professional programs are required to submit an annual review. Please follow this simple 2-step process: Step 1:Complete your review by inputting your information directly into the PATH tool, then follow the established 2015 Program and Unit Review Process and Timeline Step 2: TheVCAA will have someone copy your information over from the PATH tool into the ARPD online tool. To see how your review flows through the colleges approval and planning process, see the 2015 Program and Unit Review Flowchart

25. Process for Completing the Comprehensive Instructional Program Review Click on the link below to find out if you are scheduled for a comprehensive review this year 3-Year Comprehensive Program Review Schedule Supplementary training will be provided at a later date for all scheduled programs

26. Role of Document Steward • In order to better manage documentation used in the program and unit review process this year, we have created a new role known as Document Steward. • The role seeks to better define who is responsible for updating documentation, ensuring the there is some kind of governance/sign-off taking place, version control, and being the keeper of the original copy. • Going forward, CERC will survey the campus on areas for improvement to the program and unit review process and develop an action plan. • From the action plan the CERC will forward recommendations to the document stewards of interest. • The document steward will update their documentation prior to summer break so that all documentation has been completed for the year before starting a new cycle. Current document stewards are: IRO: P/U Process documentation incl. flowchart, narrative, and timeline. IAC: Schedule updates and program and unit annual and comp templates VCAA: CERC Process

27. Process Owners • The graphic below depicts who is responsible for each part of the program review process—from Assessment through CERC. For more information on the processes please use the contact list below. Contact List Assessment: Reshela DuPuis 934-2649 ARPD: Shawn Flood 934-2648 Annual/Comp Process: Joyce Hamasaki 934-2522 CERC: Joni Onishi 934-2514

28. Quantitative Indicators Table Organization • Health calls: Demand, Efficiency, Effectiveness, Overall • * -- Data element used in calculation of health call calculation • Last Updated date • “Not yet reported” measures • #18a Persistence fall to fall zero percent? Will populate after this fall’s census date • Distance Education • UHCC Performance Funding – your programs contribution

29. Terminology / Timing • The Census freeze event is the fifth Friday after the first day of instruction. • The End of semester (EOS) freeze event is 10 weeks after the last day of instruction. • Degrees are conferred in the “Fiscal year”. The fiscal year value represents the ending of that fiscal year. For example, a FISCAL_YR_IRO value of 2013 indicates the fiscal year 2012-2013 (July 1, 2012 to June 30, 2013) which includes Summer 2012, Fall 2012, and Spring 2013 semesters. • “Home Institution” is the campus where the student was admitted. • “Program Year” listed on your data sheet for “13-14” represents your programs data in semesters: Summer 2013, Fall 2013, and Spring 2014. • Student information data this year comes exclusively from the Operational Data Store (ODS). Organizationally this means that all community colleges are getting the data from the same place and at the same snapshot in time (this is a good thing).

30. #1 Number of Majors • Count of program majors who are home‐institution at your college. • Count excludes students that have completely withdrawn from the semester at CENSUS. • This is an annual number. Programs receive a count of .5 for each term within the academic year that the student is a major. A maximum count of 1.0 (one) for each student. #1a Number of Majors Native Hawaiian Count of program majors who are Native Hawaiian and home‐institution at your college. Count excludes students that have completely withdrawn from the semester at CENSUS. This is an annual number. Programs receive a count of .5 for each term (fall and spring) within the academic year that the Native Hawaiian student is a major. A maximum count of 1.0 (one) for each student.

31. #1b Fall Full-Time • Percentage of majors (#1) enrolled in 12 or more credits in the college in the reporting Fall semester. #1c Fall Part-Time Percentage of majors (#1) enrolled in less than 12 credits in the college in the reporting Fall semester. #1d Fall Part-Time who are Full-Time in System Percentage of majors in #1c (enrolled in less than 12 credits in the reporting Fall semester in the institution) who are enrolled in credits in other UH institutions where their total number of credits enrolled in the UH System is equal to or greater than 12.

32. #1e Spring Full-Time #1g Spring Part-Time who are Full-Time in System Percentage of majors enrolled in 12 or more credits in the reporting Spring semester at the institution. #1f Spring Part-Time Percentage of majors enrolled in less than 12 credits at the institution in the reporting Spring semester. Percentage of majors in #1f (enrolled in less than 12 credits in the reporting Spring semester in the institution) who are enrolled in credits in other UH institutions where their total number of credits enrolled in the UH System is equal to or greater than 12.

33. #2 Percent Change Majors from Prior Year • In alignment with UHCC Strategic Planning Goals, General and Pre‐Professional Education programs are expected to grow by 3% per year. • Difference between the number of majors (#1) in the current year from the prior year, divided by the number of majors in the prior year. • For example, in the liberal arts program current year 1415 there were 1086 majors. In the prior year 1214 there were 1346 majors in the program. • ((CY-PY)/PY)*100 = • ((1086-1346)/1346)*100 = -19.3%

34. #3 SSH Program majors in Program Classes • The sum of Fall and Spring Student Semester Hours (SSH) taken by program majors in courses linked to the program. Captured at Census and excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Differs from MAPS because UHCC data includes Cooperative Education (93 series) as there is a resource cost to the program.

35. #4 SSH Non-Majors in Program Classes • The sum of Fall and Spring SSH taken by non‐program majors (not counted in #3) in courses linked to the program. Captured at Census and excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Differs from MAPS as UHCC data includes Cooperative Education (93 series) as there is a resource cost to the program.

36. #5 SSH in All Program Classes • The sum of Fall and Spring SSH taken by all students in classes linked to the program. Captured at Census and excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Differs from MAPS as UHCC data Includes Cooperative Education (93 series) as there is a resource cost to the program.

37. #6 FTE Enrollment in Program Classes • Sum of Student Semester Hours (SSH) taken by all students in classes linked to the program (#5) divided by 30. Undergraduate, lower division Full Time Equivalent (FTE) is calculated as 15 credits per term. • Captured at Census and excludes students who have already withdrawn (W) at this point.

38. #7 Total Number of Classes Taught • Total number of classes taught in Fall and Spring that are linked to the program. • Concurrent and Cross listed classes are only counted once for the primary class. • Excludes Directed Studies (99 series). • Differs from MAPS as UHCC data Includes Cooperative Education (93 series) as there is a resource cost to the program.

39. Liberal Arts Program Scoring Rubric Definitions Your program health is determined by 3 separate types of measures: Demand, Efficiency, and Effectiveness. This slide explains why these measures were chosen to determine program health. • Demand: A seeking or state of being sought after. i.e. your programs ability to attract new students every year based on your offering. • Efficiency: Acting or producing effectively with a minimum of waste, expense, or unnecessary effort. i.e. your programs ability to use its resources in the best possible way. • Effectiveness: Stresses the actual production of, or the power to produce an affect. i.e. your programs ability to produce the desired result.

40. Determination of Program’s Health • Again this year, the system office will calculate and report health calls for all instructional programs using the most recent program year data. • If you are interested in how the Liberal Arts Health Calls were determined, refer to the rubric link available on the ARPD website.

41. #8 Average Class Size • Total number of students actively registered in Fall and Spring program classes divided by classes taught (#7). Does not include students who have already withdrawn from the class by Census. • Excludes Directed Studies (99 series). • Differs from MAPS because UHCC data includes Cooperative Education (93 series) and there is a resource cost to the program. #9 Fill Rate Total active student registrations in program classes (number of seats filled) at Fall and Spring census divided by the maximum enrollment (number of seats offered). Captured at Census and excludes students who have already withdrawn (W) at this point.

42. #10 FTE BOR Appointed Faculty • Sum appointments (1.0, 0.5, etc.) of all BOR appointed program faculty (excludes lecturers and other non BOR appointees). • Uses the “hiring status” of the faculty member – not the teaching/work load. • Uses the Employing Agency Code (EAC) recorded in the Human Resources (HR) database to determine faculty’s program home. • Data provided by UH Human Resources Office as of 10/31/2014. • Faculty Teaching solely Remedial and/or Developmental Reading, Writing, or Mathematics are excluded from the counts in Liberal Arts and included in the counts for the Remedial/and or Developmental programs. Faculty with “split” assignments among Remedial/Developmental classes and Transfer level classes are reflected proportionally. • If your FTE BOR faculty count is off for your program, contact the VCAA. • Click here for the count of BOR Appointed Program Faculty in your program: ARPD Faculty to Program

43. #11 Majors to FTE BOR Appointed Faculty • Number of majors (#1) divided by sum appointments (#10) (1.0, 0.5, etc.) of all BOR appointed program faculty. • Data shows the number of student majors in the program for each faculty member (25 majors to 1 faculty shown as “25”)

44. #12 Majors to Analytic FTE Faculty • Number of majors (#1) divided by number of Analytic FTE faculty (#12a). #12a Analytical FTE Faculty (Workload) Calculated by sum of Semester Hours (not Student Semester Hours) taught in program classes divided by 27. Analytic FTE is useful as a comparison to FTE of BOR appointed faculty (#10). Used for analysis of program offerings covered by lecturers.

45. #13 OverallProgram Budget Allocation • The overall program budget allocation = General Funded Budget Allocations (13a) + Special/Federal Budget Allocations (13b) + Tuition and Fees (13c) + Other fees • The overall program budget allocation is automatically calculated when you enter your general funded budget allocation, special/federal budget allocation, other funds, and/or tuition and fees, into the online tool tab called, “Cost per SSH.” Note: Joni will have the budget data uploaded for you this year. • The overall program budget allocation is to be determined by the College using these guidelines from VCAA/DOI/ADOI and should include: Salaries (general funds, special funds, etc.), overload, lecturers, costs for all faculty and staff assigned to the program, supply and maintenance, amortized equipment, and tuition and fees.

46. #13a General Funded Budget Allocation • The general funded budget allocation = actual personnel costs + b budget expenditures #13b Special/Federal Budget Allocation The dollars from Federal grants #13c Tuition and Fees The amount collected for tuition and fees in the 2015 academic year.

47. #15 Number of Low Enrolled (<10) Classes #14 Cost per SSH Overall Program Budget Allocation (#13) divided by SSH in all program classes (#5) • Classes taught (#7) with 9 or fewer active students at Census. • Excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Includes Cooperative Education (93 series) as there is a resource cost to the program.

48. #16 Successful Completion (Equivalent C or higher) • Percentage of students actively enrolled in program classes at Fall and Spring census who at end of semester (EOS) have earned a grade equivalent to C or higher. #17 Withdrawals (grade = W) Number of students actively enrolled (at this point have not withdrawn) at Fall and Spring census who at end of semester have a grade of W.

49. #18 Persistence Fall to Spring • Count of students who are majors in program at fall census and at subsequent Spring semester census are enrolled and are still majors in the program. • Removed from the count (both numerator and denominator) are program major students to whom a program degree (or CA if highest credential awarded) has been conferred in the reporting Fall semester. • Example: 31 majors start in Fall 21 majors of the original 31 persist into Spring 21/31 = .6774 or 67.74%

50. #18a Persistence Fall to Fall • Count of students who are majors in program at fall census and at subsequent Fall semester census are enrolled and are still majors in the program. • Removed from the count (both numerator and denominator) are program major students to whom a program degree has been conferred in the first Fall, Spring or Summer reporting term. • Example: 31 majors start in Fall 11 majors of the original 31 persist into Fall of the next year 11/31 = .3548 or 35.48%