With thanks to David Leyton-Brown, former Executive Director OCGS

1. KEY ELEMENTS IN THE APPRAISAL BRIEFWilfrid Laurier University, Faculty of Graduate StudiesNovember, 2008 With thanks to David Leyton-Brown,former Executive Director OCGS

2. Types of Appraisal Briefs • Standard Appraisal Brief: for new programs • Likely to remain the same after transition to new review agency • Periodical Appraisal Brief: for existing programs • Template likely to change when transition to new review structure, but business as usual until 2010 • Abbreviated brief if program less than 3 years old • New Fields: request may be submitted any time

3. OCGS review process • Brief sent to OCGS by the graduate dean on behalf of the university after all internal approvals, including Senate, obtained. • OCGS sends brief to a multidisciplinary Appraisal Committee composed of senior faculty members experienced in graduate education (nominated by universities and approved by Council) • Appraisal Committee identifies concerns/issues; selects consultants and OCGS informs the university

4. Process (cont’d) • University gathers additional material for consultants to address these issues or provide information on new developments • Consultants visit: usually a day or a day and a half • Investigate issues and others they’ve identified • Meet with grad dean first, then all stakeholders • Write and submit report to OCGS • Report is sent to the university; we respond to all issues • Rating of Good Quality, Good Quality with Report, Conditional Approval, Not Approved • We can offer and receive MTCU funding only for approved programs

5. Unit of Analysis • Appraisal of the Graduate Program • Not the administrative unit (i.e. department, school) • Only three levels of analysis in the appraisal process • Program • Field • Individual (professor, student, course, etc.)

6. Template • Consult the bylaws, follow the template and use my checklist • Ric Knowles’ notes are also helpful • Not a previous brief (significant changes in bylaws in the last 2 years)

7. “Truth in Advertizing” Key concept in OCGS reviews • If a student enters the program because of the promise of strength in a particular area of interest, that strength must be offered in the program • Strength seen in: • Faculty • Courses • Fellow students • Intellectual activities

8. The Program • A coherent program of study and research • Academic content and appropriate requirements • Curriculum • Show how it is different from undergraduate program (e.g., master’s program shouldn’t look like the 5th year of an undergraduate degree)

9. Research Component • A graduate level research component is essential • Must be demonstrated • Analytical/interpretive skills • Including in: • professional programs • coursework-only programs

10. Faculty Provide intellectual leadership in the disciplinary area(s) of the program • Research active • Peer reviewed publication • Research funding • Graduate education involvement • Membership in Faculty of Graduate Studies • Supervision – experience and activity • Program activities (intellectual community)

11. Faculty Data • CV (OCGS format or approved alternative) • Faculty By Field Table • Categories for each core member of the program • Do not need to include CAS instructors in the tables, but should in the narrative

12. Fields: optional in master’s; required in doctoral programs For appraisal purposes any of these are fields, whatever term is used internally: • Streams • Subfields • Areas • Sub-areas • Concentrations • Specializations • Themes • Options • No such thing as a subfield • If it is an area or strength in the program that the university wants to advertise, for appraisal purposes it is a field

13. Faculty Complement How many faculty per field? • Critical mass – supervision, teaching, intellectual community • 1 is not enough • 2 is not enough • 3? 4? 5? …. • No benefit in maximizing the number of fields

14. Consequences of field identification • CAN include a faculty member whose expertise is not in one of the fields • CAN offer a course outside the fields • CAN permit a student to write a thesis on a subject outside the fields • But not explicitly recruit the student to do so • CAN name each faculty member’s expertise • CANNOT advertise other program “strengths”

15. Outcomes • Learning objectives and learning outcomes • Student data • Completion rates and times to completion • External scholarships and awards • Outputs – productivity, quality, success • Remember to keep collecting these data: • Publications – from thesis research • First employment or further study • Awards

16. Objectives (Bylaw 10.1.2) • Degree level expectations (how demonstrated?) • Nature of graduate education experience • Intended career preparation • Learning objectives • What do you want your students to learn or be able to do by the time they complete the program? • How will you know if they have/can?

17. Undergraduate-Graduate Courses (Bylaw 10.4.4) “The number of undergraduate courses, or combined courses in which undergraduate students predominate, should be no more than one third of the total course requirement for the degree” • Program requirements – i.e. maximum number of combined courses • Course enrolment data • Number of graduate courses offered

18. Consultants • In all standard and periodic appraisals for the next year, at least. • Benefits - provide feedback • 2 for master’s • Normally 2 (perhaps 3) for doctoral • Committee’s questions to consultants • Helpful to provide material to consultants (and Committee)

19. Consultants’ Report Response to consultants’ report(s) • University response, not program response • Respond – don’t cherry-pick or cheerlead • Address all the consultants’ points • Address Committee questions (if needed)

20. Interdisciplinarity • Assure faculty availability • Effect on home program(s)? • Truth in advertising • Deliver what you claim • Financial support • Especially TAs – spillover effects?

21. Part-Time • Section 10.4.3 • Principle – content and educational experience should not be inferior to that for full-time students • Time path – mapping progress through the program

22. Admission Requirements • Section 10.4.2 • Research-intensive or course master’s • Honours degree – four year • Professional master’s • Four year degree or equivalent • Exceptional admission • State criteria • Report on proportion admitted exceptionally

23. Irritants to appraisal committee • Denial • Concerns expressed in the previous appraisal • Major issues – elephant in the room • Sloppiness • Tables and text don’t match • Required data incomplete

24. Special Situations1. Abbreviated Brief (18.0) • Usually no consultants unless OCGS feels there’s a problem: then one consultant • Describe program and provide calendar material • NEW enrolments, graduations, withdrawals etc. by year; funding, thesis topics, employment • Describe programs and fields • List core faculty by field: • Highlight changes and the impact on the program • Include CVs of new faculty members

25. Special Situations2. New Fields (28.1.4 a) • OCGS will look for viability of new fields given existing fields (i.e., enough faculty, courses) • Describe intellectual focus of new field • Core faculty by field table (and CVs) • Courses in new field • Relationship of new field to parent program • Impact on existing fields (effect of diversion of faculty, expected new hires, involvement of other university faculty, etc.) • A problem to give field same name as program