WEBINAR FOR ACBSP FIRST-TIME EVALUATORS Presented Spring 2013

1. ACCREDITATION COUNCIL FOR BUSINESS SCHOOLS AND PROGRAMS WEBINAR FOR ACBSP FIRST-TIME EVALUATORS Presented Spring 2013

2. Purpose of this webinar: • Provide additional information for our first time evaluators. • Provide additional information on how to write strengths and opportunities for improvement comments. • Provide an opportunity for you ask any questions you may have.

3. Agenda: • Welcome and Introductions • Role of the evaluator • Resources available to site visit team • Scheduling of Consensus Calls and your role during the call • Travel arrangements and travel policies • Communication with institutional contact • Site Visit Schedule • Format for final comments to be included in the feedback report • Review and submission of feedback report • Submission of expense reports and other forms • Any other topics

4. ACBSP STAFF CONTACTS: Steve Parscale Director of Accreditation 913-339-9356, Mobile: 913-526-5285 sparscale@acbsp.org Diana Hallerud Assistant Director of Accreditation 913-339-9356, Mobile: 816-699-9964 dianahallerud@acbsp.org

5. ACBSP STAFF CONTACTS: Ron DeYoung, Executive Liaison for the Baccalaureate/Graduate Degree Board of Commissioners, DEYOUNG@nwmissouri.edu Mobile: 816-695-9585 Larry Zachrich, Executive Liaison for the Associate Degree Board of Commisisoners, lzachrich@acbsp.org Mobile: 419-966-7091

6. ACBSP STAFF CONTACTS: Maria Hallerud, Accreditation Manager, mhallerud@acbsp.org Office: 913-339-9356 Andrea (Andy) Ernst, Accreditation Administrative Assistant aernst@acbsp.org Office: 913-339-9356

7. Primary Role of the Evaluator: • Evaluate and validate the information provided in the self-study to the standards and criteria. • Provide feedback ready comments to the business unit regarding strengths and opportunities for improvement based on information from the self-study and observations during the site visit.

8. Travel arrangements: • Review the Travel Guidelines • Evaluators should plan on using their personal credit cards to pay for airfare and be reimbursed through ACBSP. • There are exceptions for international travel, when the institution may choose to make the flight arrangements for you or when you can use the ACBSP travel agent. You should always check with Steve Parscale or Diana Hallerud before contacting the ACBSP travel agent. • The institution will typically make the hotel reservation for the site visit team, but it varies by institution for the billing arrangements. The institution sometimes sets up a direct billing arrangement. Most institutions ask the site visit team to pay for the hotel room directly and seek reimbursement through ACBSP. • Travel expenses are processed on the 15th and the 30th of the month. • Submit your expense forms and receipts to Mary Riley via email at maryriley@acbsp.org, fax to 913-339-6226, or send snail mail.

9. Resources available to you, cont. Files on the CD and notebook you received Under Tab One: Evaluator Training PowerPoint Slides – June 2011 Scoring Guidelines & Worksheet Practical Guide for Evaluators (includes) ACBSP Observation Card ACBSP Site Visit Notes Do’s & Dont’s for Evaluators Site visit Issue Worksheet Site Visit Questions Site Visit Issue Strategy

10. Resources available to you, cont. Files on the CD and notebook you received Under Tab Two: Process Manuals Under Tab Three: Standards and Criteria Under Tab Four: Evaluator Workbook Feedback Report Executive Summary Template

11. Resources available to you, cont. Files on the CD and notebook you received Under Tab Five: Travel Reimbursement Guidelines Travel Reimbursement Form Travel Expense Form Evaluator Agreement Form Evaluator Critique Form

12. Review Process: • Review contents of package and verify that you have received items checked on the checklist. • Read the entire self-study, including the overview, prior to the consensus call. The overview will provide important information regarding mission, vision, student demographics, strategic challenges, etc.

13. Reviewing the self-study, cont. READING THE ENTIRE SELF-STUDY DOCUMENT IS IMPORTANT BECAUSE • a standard or specific criteria may not be addressed where you expect to see it. Don’t be surprised if you see student satisfaction results in Standard 4 rather than Standard 3 or Standard 4 items such as learning outcomes in Standard 3 or Standard 6. • the standards have many linkages and information can be found throughout the document. Some self-studies even have most of the results in the appendices rather than the self-study document. • the information may not be in the correct place, but the business unit can still meet the standard. It just requires additional effort on your part to locate the information. • Using post-its or different colored highlighters when you find information referencing the standards you are responsible for can be helpful when you begin preparing your comments.

14. Review Process: • Use the evaluator workbook and begin writing observations prior to the consensus call. • The team chair will notify each team member regarding the standards you will have primary responsibility for reviewing and providing strengths and OFI’s. The chair may ask you if you have a preference based on your expertise.

15. Contact with the Institutional Coordinator/Contact • All communication with the institutional coordinator or contact should be routed through your team chair. • Any requests for clarification or additional information should be forwarded to the team chair. • This way the institutional coordinator is not responding to duplicate requests for information. Your cooperation is appreciated.

16. The Consensus Call: • Using the scoring worksheet, evaluate each of the standards using the best in class to major improvements needed criteria. • The team chair will review site visit logistics and lead the discussion regarding observations after reviewing the self-study.

17. The Consensus Call continued. • For baccalaureate/graduate degree site visits, the commissioner and the mentor are invited to participate in the call and answer questions. • For associate degree site visits, the mentor is invited to participate in the consensus call.

18. The Consensus Call continued. • The team will identify site visit issues and additional information needed to be supplied prior to or during the site visit. • Discussion regarding meetings and the site visit agenda will be finalized.

19. Overall Scoring Summary Worksheet • Use these scoring dimensions to complete the table on the following page. Your individual review will be discussed during the consensus call.  • Individual Assessment Dimensions • Best in Class • Good to Excellent • Good • Major Improvements Needed • Poor

20. Scoring Band

22. Scoring Summary worksheetBaccalaureate/Graduate Degree:

23. Typical Site Visit Schedule: • Review the typical schedule outlined in the Practical Guide for Evaluators. • Typically, team members arrive Sunday afternoon/early evening and have a team meeting to discuss site visit issues and finalize the schedule. • Monday is typically filled with meetings with administrators, faculty, students, advisory boards, assessment officer, etc. • Most teams complete all of the meetings on Tuesday morning and hold any follow-up meetings early afternoon. Tuesday afternoon and evening will be used to finish writing your portion of the report. • Your completed portion of the report should be given to the team chair to be used to prepare the executive summary and as a part of the exit interview.

24. CONTINUOUS IMPROVEMENT PROCESS APPROACH IMPROVEMENT DEPLOYMENT OR IMPLEMENTATION RESULTS and LEARNING

25. Keep in mind: • When describing processes for the business school or programs, that the overall value of the results and improvements taking place is dependent on the approach (what they do) and the level and extent of deployment (how they do it).

26. Review Process after Consensus Call: • Continue to refine comments for the feedback report prior to the site visit – share with the team chair. • Ask for guidance when needed – there are no dumb questions and every chair has experienced a first visit – he or she will welcome any questions you have.

27. MOST IMPORTANT THING TO REMEMBER AS AN EVALUATOR: • A site visit that is conducted in an organized and professional manner followed by a feedback report that is professionally and clearly written will create a positive impression of what ACBSP represents as an accrediting body. • While you may travel to a location new to you, participating in a site visit is not a vacation and should not be considered as such.

28. AS AN EVALUATOR, DO… • Remember that you serve as an ambassador for ACBSP. Most of the faculty and staff you will be interacting with may not have an opportunity to attend regional or national conferences. The site visit experience will be what they base their perception of ACBSP on. Make it a positive one! • Come prepared for the site visit with materials read in advance. You are part of a team with your team members depending on you. You will work hard during the site visit and will find it immensely rewarding. • Say thank you and complement the institution for their hospitality and availability of materials. Many institutions provide resources and snacks over and above what is expected. • Make every attempt to stay on the established schedule and let the next person know in advance if the team is falling behind schedule. • Let the team chair contact the institutional contact if additional information or resource is needed. A little diligence on your part will result in a stronger report and assist the Board of Commissioners in making the accreditation decision.

29. AS AN EVALUATOR, DO… • Let the team chair serve as the primary point of contact in making the arrangements both in advance and during the site visit. • Contact the ACBSP office if you encounter any problemsand are not sure how to proceed. • Remember that each institution has its own mission and operating processes and procedures. You’re only concerned with how the processes and procedures used serve the institution, not how they compare to your institution. • Remember that the size and location of the institution you are visiting will determine the type of lodging available during your stay. Institutions are asked to provide reasonable, clean, comfortable accommodations as close to the campus as possible.

30. AS AN EVALUATOR, DO NOT… • Market your services as a consultant or as a potential recruit at the institution. The evaluator’s role is to evaluate and validate the information in the self-study, not to offer personal assistance when a gap in a particular area is observed or to look for employment opportunities while on campus. • Under any circumstances, attempt to recruit faculty as either a student or employee from the institution you are visiting. Even if a faculty member voices an interest in leaving the institution and there is a position available at your institution, do not discuss this opening with the faculty member. Faculty members who express an interest in seeking additional degrees should be directed to ACBSP for a list of institutions available rather than to your specific institution. • Cancel appointments with administrators unless you can provide a valid reason for the cancellation. Usually the President or Vice-Presidents work their schedules around the site visit schedule and do not appreciate a sudden cancellation.

31. AS AN EVALUATOR, DO NOT… • Under any circumstances, make negative comments regarding the business faculty or administration to anyone during the site visit. • Criticize the content or quality of the self-study during the site visit. If a standard is not adequately addressed, let the institutional contact know what is missing and ask if the information can be derived from other sources. • Wait more than 10 days to submit the feedback report to ACBSP.Do not submit the report unless you proofread the report thoroughly before submitting to ACBSP. All team members should have the opportunity to read the final report prior to submitting the final report. Please make sure the comments are complete and accurately reflect your observations as an evaluator.

32. Writing Comments: • THINGS TO REMEMBER: Comments are based on the requirements of the standards and criteria, not personal opinions or how processes are followed at other institutions. Comments should be clear and concise and should address the situation, how the business unit does (for strengths) or does not meet (for OFI’s) the standard, and explain the significance of the situation. Comments should not be prescriptive unless you are using language directly from the standards and criteria. Comments should be written in complete sentences and should use proper grammar and punctuation. Avoid the use of slang or local expressions others may not be familiar with, especially when on international visits. Example: “pig in a poke”.

33. Writing a strength: If the institution has a strong strategic plan that includes both short and long-term goals and has a good approach, has been deployed or implemented, demonstrates learning, and identifies improvements, your comment could state: • “The John Doe School of Business has a strategic plan that has identified both short and long-term goals. The plan is systematic with an approach that has been implemented for all disciplines, provides examples of learning and evidence of improvements the school has made. The strategic plan helps the school to plan for increasing enrollment and the replacement of faculty members as they retire.”

34. Writing a strength: If the institution has an assessment instrument and provides evidence of a systematic process and examples of how the results are used to make improvements, you could provide a comment such as: • “The John Doe School of Business has administered the MFT test for all disciplines for five years. Each department documents their review of the pre and post results each semester during department meetings and makes improvements based on the results. For example, the accounting department added additional projects for preparing financial statements in the ACCT 3300 course.”

35. Writing an opportunity for improvement: Suppose the institution has identified: • increasing enrollment and a large number of faculty scheduled for retirement as strategic challenges. • All the strategic objectives in the strategic plan are for one to three years, and faculty are not consulted in the development of the objectives. The opportunity for improvement could read, “While the John Doe School of business has strategic objectives for up to three years, there was no evidence of planning beyond three years or involvement by faculty in the development of the objectives. Developing long term goals and involving more stakeholders such as faculty could provide resource planning to meet the increasing enrollment and replacement of faculty members scheduled to retire in the next five years.

36. Writing an Opportunity for Improvement: Suppose the institution administers the Major Field Test (MFT) to all students as a pre and post test. • There is no evidence, either in the self-study or on-site, that the results are reviewed to indicate that there is learning or subsequent improvements are made. In addition, the school does not use the data for comparative purposes Your opportunity for improvement comment could read: “The John Doe School of Business provided three years of data for the Major Field Test (MFT). However, there is no evidence that the school is using this data to measure how their students are performing in each discipline and how the results compare to other institutions with a similar mission. Analyzing the results to identify program strengths and weaknesses could allow the school of business to make improvements where needed and provide an external source of comparative data.”

37. Submitting your final comments: • Using your initial review, you will conduct interviews and review documentation to validate the information provided in the self-study. You may find that some initial strengths become OFI’s and some OFI’s will either be reduced and/or become a strength. Wording for OFI’s should change from “it is not clear” to “there is no evidence” following the site visit. Before submitting your comments to the chair, read them from the perspective of someone who did not write the self-study. Based on the comment, would the person understand the jist of the comment? Can they relate it to the standard or criteria you are referencing? Ask your team members for help if you get stuck. They won’t mind and will be asking you to read a comment for clarity as well.

38. Final Steps: • On Tuesday evening, you will submit your final comments to the chair and confer with the team if you have questions. • Wednesday morning, you will participate in the exit interview and return home. • The chair should send the report for a final review to team members before submitting it to ACBSP. Respond with your comments promptly and let the chair know if you saw evidence of meeting a standard that a team member may have missed or if a comment is not clear to you. • Submit your expense report, receipts and evaluation form to ACBSP as soon as you return from the trip.

39. Questions? • While you will work hard, most evaluators find the site visit experience to be very rewarding and no matter how many visits they have participated in, they always learn something new that they can use to improve the processes at their own institution. • Thank you for participating in this webinar. If you have any suggestions for improvement, please feel free to contact me by phone or via email at dianahallerud@acbsp.org. • We hope you enjoy your first site-visit experience.