Understanding the Evaluator’s Responsibilities

1. Understanding the Evaluator’s Responsibilities Karen Froslid Jones Director, Office of Institutional Research and Assessment American University

2. Overview 1. Accreditation and the Visit 2. The Role of the Chair and Evaluators 3. Preparing for the Visit 4. What to Expect During the Visit 5. What Happens After you Leave 6. Keys to a Successful Evaluation

3. Participants in the Accreditation Process Public Institution Middle States Rep Self-Study Committee President Faculty, administrators Staff and Students Commission Team Chair Visiting Team

4. Putting The Visit in Context:The Road to Re-Accreditation Before Visit During Visit After Visit • Chair sends report to • institution • Institution corrects • mistakes of fact • Chair sends final report • Institution sends response • Chair attends commission • meeting • Commission acts on • accreditation • Institution – and team- • is notified of commission • action. 2004-2007 • Team meetings • Reception/Dinner • Meetings with • Steering Committee, etc. • Interviews • Writing time • Report written • Recommendation on • re-accreditation • Oral Report • Self-Study Institute • Chair and Steering Committee Selected • Self-Study Design Submitted • Visiting Team selected • Self-Study Completed • Visit to campus by MS Rep • Logistics decided • Materials Sent to Team • Additional documents requested • Travel info communicated.

5. Purpose of Visit • The importance of peer review. • The value of meeting face-to-face. • The opportunity for institution to benefit from other’s expertise. • The need for objective assessment.

6. The Value of an Evaluator • Colleague • Similar institution? • Similar mission? • Appreciation of the challenges facing institutions • Expertise • Particular Area(s) • Characteristics of Excellence/Middle States Expectations • Provide Outside Perspective • Not tied to politics/personalities/history of institution • Able to look at ‘big picture’ and put it into context • No conflict of interest

7. The Role of the Chair • Acts as conduit between visiting team and institution. • Visits Institution before team visit • Gets early copy of Self-Study • Represents team when planning logistics • Resolve problems/conflicts between team and institution • Main communicator between team and institution during visit • Represents team at Commission meeting

8. The Role of the Chair (cont.) • Organizes and Leads Work of Visiting Team • Sets team member assignments (primary and secondary) • Sets up some meetings with campus groups in advance (board of trustees, steering committee, etc.) • Sets report format • Sets work agenda/deadlines • Is draft required? • When will team meet? • When is ‘final’ report due to chair? • Leads discussions • Leads/Organizes Exit Interview

9. The Role of the Evaluator • Assigned one or two primary areas (usually connected to the Characteristics of Excellence) to review. • Responsible for reading the self-study, conducting interviews, and reviewing documents to determine the degree to which Characteristics are being meet. • Responsible for writing portion of final report. • Responsible for contributing to overall report.

10. Before the Visit: What to do Now • Know Characteristics of Excellence • Be prepared on standards you consider your areas of expertise • Know all standards thoroughly • Look for ways that standards interact • Familiarize yourself with self-study options – be sure you understand the type of self-study your institution is using • Know the Report Process/Possible Actions • Begin to familiarize yourself with institution • Check out website

11. Before the Visit: What to do When you Receive the Report • Read entire report several times at least a few weeks before visit. • Clarify with chair your expected areas of coverage. • Decide on ‘must’ interviews – inform chair • Any documents you need right away? Let chair know. • Provide institution/chair with travel plans.

12. Arriving on Campus • Accommodations • Usually stay at same location, close to campus • Leave plenty of time to get checked in/settled before first meeting. • Transportation • Meeting Room(s) • Usually a work room/ meeting room set up in central location

13. Arriving:The Atmosphere on Campus • Keep in mind that this is a BIG deal • Welcoming • Nervous • Worried • Excited • Agendas • Protective • Defensive • Helpful

14. The Visit Begins • 1st Team Meeting • Discuss logistics/assignments/report format. • Come prepared to discuss initial impressions. • Have preliminary interview needs ready. • Host’s Program • Chance to get to know individuals/’players’. • Convey interest/enthusiasm for visit. • Informal inquiries/ not hard line questions. • Be aware of time

15. Setting Up Interviews • Read report (and other documents) with eye towards people you may want to meet. • Who seem to be a key players in the area you are covering? • Do certain names come up in the report? • Don’t be afraid to ask ‘who would know about_____?’ • Prioritize and Coordinate • Can ask interviewees if they can suggest others that may be worth meeting.

16. Conducting Interviews • Have a clear objective going in. • What needs clarification? • What is missing from self-study and needs to be answered? • What has happened since self-study was written? • Be open to where the interview may go. • Don’t assume everyone you interview knows about accreditation. • Create a friendly, open conversation. Avoid overt judgments. • Don’t make promises/commitments. • Look for evidence of a culture of ‘continuous improvement’. • Example: • What did you learn about your institution as a result of doing the self-study? Did any findings surprise you? • How do you plan to implement the recommendations of the self-study? • Ask questions that give you a sense of direction/commitment/focus

17. Conducting Interviews (cont.) • Keep very good notes. • Get cards, if possible, from interviewees. • Consider meeting them on their ‘home turf’. • Consider how combinations of individuals together may change dynamics of interviews.

18. Tips for Interactions with Institution • Be careful of off-hand remarks • Everything you say/do will be interpreted • Be neutral/open • Listen • Observe • Reserve judgments/opinions • Smile

19. Using the Document Room • Make sure you build in time early to view documents. • Come prepared with a list of ‘must see’ documents. • Be open to looking through a wide range of information. • Meeting notes, school newspaper stories, survey results, annual reports, etc. • Put things back where you found them! • Shred – or take home - sensitive notes. Don’t forget to erase info from borrowed computer!)

20. Team Members Informal Observation Writing the Report Findings Significant Accomplishments Suggestions, Recommendations, Requirements Characteristics Institution Mission Document Analysis Individual Interviews Self-Study TEXT

21. Writing the Report* DO • Speak directly to the accreditation standards and institution mission • Respond to/recognize the approach used in the self-study • Concentrate on the fundamentals • Cover all areas of activity, including off-cite, certificate programs, etc. • Provide rationale and evidence for the team’s recommendations • (model effective institutional assessment) • Make clear distinctions between suggestions, recommendations and requirements. • Emphasize the importance of outcomes and assessment of both overall institutional effectiveness and student learning. DON’T • Be highly prescriptive. • Name individuals. • Cite specific formulas, solutions, educational theories, etc. • Reveal accreditation recommendation. * See pp.46-49, Team Visits

22. Writing Tips • Clarify which ‘voice’ you will use as a team. • Begin writing early – you can always revise. • Keep track of who you talked to/when • Keep track of sources of factual information (keep citations) • Integrate report as much as possible.

23. Team Interactions • A friendly, collegial atmosphere • “Where you stand depends on where you sit.” • Ask questions, seek input from others. • Offer insight from your perspective. • Disagreements likely but usually resolvable.

24. Completing the Visit • The final draft • Be open to last minute changes • The oral report • Leaving the institution

25. After the Visit: What Happens Next? • Team Chair submits report to institution for review. • Institution corrects factual errors. • Team Chair submits final team report along with a confidential memo to the Commission. • Institution sends team members final team report. • Institution can provide formal response to report. • Chair Attends commission meeting. Commission acts on report. • Logistics: expense reports, evaluations, etc.

26. Miscellaneous Tips: (What you won’t find in any official publication) • Come to the visit rested • Leave home institution at home • Where comfortable/sensible clothing/shoes • Alert the institution to any food allergies, special needs, etc. • Bring a flash drive. • Expect the unexpected – be prepared to go with the flow • Don’t try to preach/perfect. • Have fun!