Department Reviews:

1. Department Reviews: What are they? Vantage points: the reviewed and the reviewer. How can they be an integral part of program assessment? Mary Savina – February 24, 2009 Building Strong Geoscience Departments – Workshop on Program Assessment

2. Acknowledgements • My colleagues in the Carleton College Geology Department • My colleagues in the faculty development community, at Carleton and elsewhere • The departments who’ve asked me to help review them • My colleagues who have served with me on review teams • The P. I.s of the Strong Geoscience Department program – funded by NSF

3. What’s a departmental review? • Formal – involves external reviewers • Comes up every five – fifteen years • (Generally) part of an institutional plan to review all departments and programs periodically • Results – conclusions, recommendations to the institution and to the department/program • How many of your departments have reviews like this?

4. Department reviews and assessment • Regular program reviews may predate the present concern with assessment • This is a two-edged sword: • Rich source of longitudinal information, but. . . • Review guidelines and procedures not designed around objectives and outcomes (little emphasis on assignments and student work, for instance) • Reviews not well integrated into institutional assessment/accreditation: “It remains a mystery to me why our accreditors did not value this process as strong evidence of a ‘culture of assessment.’” S. Moshier, Wheaton College

5. Logistics for review • Overall calendar • Choosing the review team(s) • Typical external review team schedule • Responsibilities of internal review team

6. The meat of the self-study: guidance from on high at Carleton • Use the eleven questions (nine of them are about curriculum/courses) • What are we happy with? • What can be improved? • How and how well are we helping students meet the College’s goals for student academic learning (Assessment Plan)? • What are we doing that we shouldn’t be doing? • What aren’t we doing that we should be doing?

7. The meat of the review (the reviewed and the reviewers) • What can a review do for us? – specific questions, in addition to the general institutional questions, resources • What can our review do for this department? • What should our self-study do? • What does the review team want to see in the self-study? • What are the main issues? • We need to read the culture of this place fast so that we can find and approach the main issues .

8. The meat of the review, continued • What are the strengths of our program? • What are the under-recognized or under-articulated strengths of their program? • How can we best respond to the review? • How can we phrase our review as concretely as possibly so as to help the department?

9. Examples of “real issues”The major • The major requirements may be too heavy, restricting student numbers to those few students who are definitely pre-professional • symptom: the students sound like addicts – they know something’s wrong with the program, but they can’t stay out of it • The major requirements may make it impossible for students to study off-campus, e.g. (too hierarchical, too many requirements) • symptom: students drop the major in order to have perfectly reasonable college experiences • The major is overly complex and hard to decipher, reducing students’ ability to come late to the field • symptom: student stories about other students

10. Relations among faculty • Un-tenured faculty need to be protected • Symptom: Excessive enrollments/courses for junior faculty • Perceived inequities of teaching loads or teaching choice • Lack of apparent respect among faculty • Difficult integration of junior faculty and their ideas • Symptoms: Frustration on all hands • Pride, prejudice, defensiveness, ego and lack of it

11. History • Trouble getting beyond a “storied past” to respond to changes at the institution and the profession • Symptom: Emeritus professor attends department meetings • Long-term impacts of absences, illnesses, tenure denials, etc. • No history of working or talking together about program objectives and desired outcomes • Symptom: Self-study is a mess – or – self-study is clearly the work of a single person • (See Scott Bair's Advancing By Retreating)

12. Space • Inflexible teaching spaces, limiting effective uses • Symptom: Spaces are vacant 90% of the time • Layout/location of space limits sense of community (rabbit warrens, separation) • Symptom: Too many corridors, no common spaces • Inadequate or inflexible space for student, faculty and student/faculty research • Symptoms: no counter spaces; unused machines sitting around

13. Time • Faculty time devoted to secretarial and logistical duties • A “department chair does everything” mentality (I would have added money and faculty here, but these are generally addressed head-on in self-studies).

14. Isolation • Within institution, possibly even within program for individuals • Lack of understanding of the program (and sympathy for its needs) by key administrators • Within and from geoscience community • Need for reassurance • Symptom: No culture of reviews (may be institutionally) • Symptom: fear of the outside reviewers

15. Untapped resources • Potential partners and allies on the campus and beyond • Alumni relations • Under-recognized strengths

16. Final comments • There are already resources on the Building Strong Geoscience Departments web site to help departments whose first need might be to get faculty buy-in to the idea of a review. • Big question: bridging the gap between outcomes assessment and review structure

17. Next up • Steve Moshier, Wheaton College – on getting the most from the department review process • Char Mehrtens, University of Vermont – on being an external reviewer, plus generalizing to universities • Discussion of review goals and how to achieve them (small group activity)

18. A typical process (drawn from Carleton’s guidelines) • May – October, year 1 • Assemble self-study/review document • Select internal and external review committees (with the Dean and chair of ECC) • Organize reviewers’ schedules • October – December, year 1 • Complete and distribute self-study • January – February, year 2 • External review team visit and report • Internal review team report • March – June, year 2 • Department responds to report • Wrap-up meeting with Dean who writes summary

19. Reviewers (drawn from Carleton guidelines) • Internal review team: • Three tenured faculty members, including one from the same curricular division • Two students, one of whom is a major, one of whom is on the ECC (Education and Curriculum Committee) • External review team: • One senior faculty member from a Carleton-like college • One senior faculty member from a major university who has some close connection with a liberal-arts college • Another senior professional in the field • One of the three should be a Carleton graduate

20. Typical external review team schedule • Kick-off meeting with the Dean and President • Meet with the entire department as a group • Meet with each department member separately (including staff) • At least one open meeting with students (majors and non-majors) w/o faculty or staff present • Meet with other interested faculty/staff • Meals and social time • Committee caucus time • Wrap-up meeting with Dean and President • *Review team is typically on campus for two days

21. Responsibilities of internal review team • Read the self-study • Meet with whole department and individual faculty and staff • Meet with students in absence of faculty and staff • Meet with other Carleton community members, as appropriate • Submit a written report

22. What’s in a review document/self-study? • Carleton instructions to external reviewers: “the report shall include an assessment of the department’s program with respect to the major, the overall academic program of the College, and the state of the discipline.” • Eleven prompting questions at Carleton, most of which deal with curriculum; one asks about relationships to programs within and outside Carleton; another asks how the department relates to the profession and practice of the discipline. • These guidelines have not changed much in a while. • (Additional questions for graduate programs)

23. Criteria for Reviews of Departments (Carleton College) • What do you see as the main objectives of the major? How well are these being met? • What changes have been made in departmental programs since the last review? How successful have they been? • Are the range, balance and sequence of the curriculum reasonable? • Are departmental offerings sufficiently comprehensive? • Is the program overly uniform or homogeneous either in its topic, its methods, or its cultural perspectives? • How well does the department relate its responsibilities to its majors with its responsibilities for the general educational program of the College? What are the goals for non-majors taking courses in this department? Are they being met? Are intermediate and upper-level courses accessible to non-majors? • Do students get sufficient opportunity to practice the discipline? • Is there a requirement for research, i.e., for truly independent work as part of the program for the major? How is the senior integrative exercise working? • Do students acquire adequate training in necessary library or laboratory skills? in reading, writing, and speaking? • What are the relationships of the department to other programs and departments within and outside the College? Is the department contributing as much as it should and as appropriately as it might? • How does the department seem to relate to the profession, to the practice of the discipline elsewhere?

24. Self-study components • Table of course offerings and enrollments since last review • Course descriptions and syllabi • Information available for majors and non-majors (including paths through the major) • Advising • Student and faculty activities • Service to other departments and programs • What alumni do • Facilities and equipment descriptions • Faculty and staff c.v. s • * This is a minimal list* (examples of student work, statements of philosophy and values, institutional data that bears on the work of the department, alumni letters, etc. might well be included)