Kristina Kauffman, Associate Vice Chancellor Institutional Effectiveness

Program Review for Academic and Administrative Departmentshttp://www.rcc.edu/administration/academicaffairs/effectiveness/review.cfm Kristina Kauffman, Associate Vice Chancellor Institutional Effectiveness David Torres, District Dean Institutional Research

Program Review Goals • Extrinsic • Resource allocation • Improvement of student learning through the provision of systematic feedback • Intrinsic • Strengthening programs through self-improvement and self-determination • Strengthening the bonds within the college community and fostering cooperation between units • Generating continuous and ongoing dialogue about how student learning can be enhanced through program and service improvements • Evaluating the unit’s contribution to achieving the district goals and strategic initiatives

Types of Program Review • Instructional • Annual • Comprehensive • Student Services • Administrative

Types of Research Support • Data = Numbers • Information Services • Raw data from Datatel • Reports organized around regular requests such as registration statistics • Institutional Reporting • Current term enrollments • Current term FTES Distributions • Current term assignments • Scheduling patterns • Institutional Research • Past term(s) data including all in Institutional Reporting’s list above • CCCCO Referential MIS files

Types of Research Support • Information = Data in a meaningful context, organized for decision making • Institutional Reporting • Summaries or organized reports about the current term • Institutional Research • Reports on Employment in the Region • Summaries or organized reports about past terms • Matriculation reports • Reports on Student Engagement and/or satisfaction • Institutional Effectiveness • Reports to the State, Federal Government and Accrediting Agencies • Reports such as the McIntyre Environmental Scans • Discipline Program Review Documents • Information used to “lobby” administration

Types of Research Support • Analysis = Projections or qualitative observations often resulting in recommendations aligned with the college mission, community need or transfer requirements • Institutional Research • Employment projections • Institutional Effectiveness • Scenarios such as those found in the McIntyre Report • Statewide and local economic and demographic projections • Analysis of trends in Education law and politics • Analysis of trends in Accreditation requirements • Units undergoing Program Review • Each unit must determine for itself which data, information and the analysis from other respected sources is relevant for its review. • Ultimately the unit must prepare its own analysis of its unit and its future as a capstone to data, information and analysis provided by others. • Units are not expected to generate their own data or information. They are expected to be able to identify additional data or information needs and advise Institutional Effectiveness and Research of those needs. As resources permit units will be provided with data and information or Institutional Effectiveness and Research will collaborate with units in collection of data.

Instructional Program Review • Comprehensive • Goals – focuses on each discipline’s courses, pedagogy, assessment plan, and future goals and objectives related to the improvement of student learning (4 year cycle). • Forms • Data • Annual • Goals – Resource allocation and assessment update • Forms • Data

Instructional Program Review • Data for Comprehensive (con’t) • Demographic information (ethnicity, gender, age group, enrollment status) • Course Enrollments by time of day offered • Counts of sections offered • Student Retention by course • Student Success rates (two measures)

Instructional Program Review • Data for Comprehensive (con’t) • FTES generated by course • Weekly Student Contact Hours by course • Full-Time Equivalent Faculty by course • Percentages of student enrollments taught by Full-Time and Part-Time faculty by course

Instructional Program Review • Data for Comprehensive (con’t) • Class enrollment by course and term • Grade point averages and grade distributions by course and term • Student success rates (two measures) by Demographics by term • Degrees and certificates awarded, if applicable

Instructional Program Review, Data Examples

Instructional Program Review • Data for Annual • Class enrollment by course and term • Retention and Success Rates for each course • FTEF, WSCH for each course

Student Services • Goals • Resource allocation • State program goals and align future goals with the College’s mission and goals. • Collect and analyze data on key performance indicators, service area outcomes and student learning outcomes, program activities, and accomplishments. • Examine and document the effectiveness of student support services. • Develop recommendations and strategies concerning future program directions and needs (e.g. budget, staffing, and resources). • Assure the accuracy of program information. • Comply with Accreditation Standards, Federal and State law, Title 5, Student Equity, VTEA, matriculation (including prerequisite and co-requisite standards), ADA (American with Disabilities Act), and other legal or certification requirements. • Forms • Data – Developed by units

Administrative Units • Goals – Same as Student Services • Forms • Data - First year the departments were asked to develop assessment plans and begin compiling data for reasonable measures of their department’s effectiveness

Focus Group Analysis • Findings • The review process was useful because it helped units take stock of effectiveness and what resources were needed. • It moved units to be more comprehensive about identifying resources and tying those resources to programmatic needs. • Need more training on budget and data analysis • Believe improvements are being made in teaching and learning as a result of program review • Believe that the resource allocation process is critical to the fate of some improvements • Changes as a result • Vice Presidents for Business Services engaged in working with units on annual review and budget plans • More data given at the beginning of the process • Enhanced training on analysis • Increasingly simplified forms

Integration with Budget and Planning

Norco Campus Ranking Criteria

Accreditation Response • Continue with current program review, add SLOs for all programs (we have them for all courses and GE) • Need better integration with budget (Norco process was in first year of cycle)

Attitude Change: Empowerment • Units get advice about lobbying for resources • Units are provided extensive support while going through the process, particularly in the beginning • Every round of program review utilizes increasing analytical skills • Units analyze their own data to improve student learning and enrollment management • Units can confirm or refute operating assumptions

Kristina Kauffman, Associate Vice Chancellor Institutional Effectiveness