Conducting an Effective and Useful Program Review Process

1. Conducting an Effective and Useful Program Review Process Patty Francis Associate Provost Institutional Assessment & Effectiveness

2. Background Information SUNY, MIDDLE STATES, AND SUNY ONEONTA

3. SUNY Assessment • Provost’s Advisory Task Force on the Assessment of Student Learning Outcomes (November 2000) Recommendations • Implementation of campus-based Assessment of the Major (i.e., program review) effective Fall 2001 • Program reviews to be conducted every 5-7 years and to include evaluations by external reviewers and assessment data on student learning

4. SUNY Assessment (cont.) • Board of Trustees’ Resolution on Streamlining Assessment (March 2010) • Ended SUNY Assessment Initiative • Called for campuses to have in place assessment plans that meet or exceed Middle States standards and those of specialized accreditors • Implications for registered academic programs • Regular review that includes assessment of student learning and external review process

5. Middle States Expectations • Clearly articulated statements of expected student learning outcomes for all programs that aim to foster student learning and development • A documented, organized, and sustained assessment process to evaluate and improve student learning • Assessment results that provide sufficient, convincing evidence that students are achieving program learning outcomes • Evidence that student learning assessment information is shared and discussed with appropriate constituents and is used to improve teaching and learning

6. Program Review at SUNY Oneonta • Has been in place since 2001 • Originally on 5-7 year schedule, now on 7-year schedule (with externally accredited programs able to coordinate program review and self-study processes) • Observations about process • Good news: they generally get done • Bad news: quality is uneven, especially with respect to student learning assessment data

7. Expanded Role of OIAE • Annual workshops are held for programs beginning the process • Feedback to be provided on submitted reviews • To be sent to program, dean, and Provost • May require some re-submission, especially if SLO data weak/missing

8. Effective Program Review Fundamental Elements

9. Getting Started • Carefully review resources on College’s assessment website (http://www.oneonta.edu/academics/Assessment/) • Guidelines and Procedures for the Review of Academic Programs are found on website and contain a detailed description of self-study report formatting and content • In Fall 2012 Provost Thompson approved APAC recommendation that College adopt the SUNY University Faculty Senate guidelines for program review (http://www.suny.edu/facultysenate/files/Program_Review.pdf) • Discipline-specific resources

10. Key Components of Self-Study Process • Consideration by faculty of program’s mission, goals, and expected student learning outcomes • Goals include student learning, but also include teaching effectiveness, student engagement, student perceptions of program, scholarly accomplishments, service, and professional development • Development of timeline for conducting and completing self-study process • Identification of external reviewers and submission of names to dean • Can be from other SUNY institutions • Three names submitted to dean, who selects two to visit campus • Submission of budget to dean for external reviewers

11. Self-Study Sections • Title Page • Table of Contents • Introduction and General Information • Assessment of Student Learning • Student learning outcomes • Description of direct and indirect measures • Conclusions regarding student learning • Plans for change based on assessment data

12. Self-Study Sections (cont.) • Program data (Program Data Summary Table is NOT needed) • Enrollment trends over past 5 years, impact of those trends, and expected changes for future • Retention and graduate outcomes • Faculty accomplishments, FTE’s, and role of adjunct faculty • Resources (facilities, operational budgets) • Use of technology in teaching and learning • Benchmarking • Program’s response to external reviewers’ report • Appendices (only external reviewer report required)

13. Recommended Timeline • Spring before year self-study to be conducted • Faculty review of process and requirements, planning, etc. • Fall Semester • Collection and analysis of information to be included in document • October • Selection of external reviewers and submission of name/budget to divisional dean for approval • November • Verify reviewers and set dates for site-visit

14. Recommended Timeline (cont.) • March 15 • Prepare penultimate draft of self-study and distribute to faculty • April • External reviewers visit campus and submit report within two weeks of visit (visit includes meeting with divisional dean and Provost) • May • Faculty construct response to external reviewers’ report and finalize self-study document and submit to divisional dean and APIAE

15. Conclusions Meshing Reporting requirements

16. Relating Annual Report, Program Review, and Assessment Plan Processes • Annual Report – due every year, includes section on assessment of student learning • Program Review – due every 7 years, includes analysis of range of processes and outcomes • Assessment Plan (i.e., APAC) – encompasses three-year period, requires reporting on annual basis effective 2011-12; exclusive focus is student learning • Recent APAC recommendations regarding reporting of assessment results in Annual Report