Importance of Evaluator Training in ABHES Accreditation Process

WELCOME ! Accrediting Bureau of Health Education Schools (ABHES) Evaluator Training 2010

THE IMPORTANCE OF EVALUATOR TRAINING IN THE ABHES ACCREDITATION PROCESSA POLICY STATEMENT The importance of the challenge you have undertaken to serve ABHES as an evaluator, whether in the capacity of program specialist or team leader, cannot be understated. You are the eyes and ears of the ABHES Commission. You represent ABHES; not only as an evaluator, but given the crucial importance of the evaluation process to accreditation itself, what you say and do and your abilities as an evaluator is reflective of ABHES. ABHES is committed to ensuring each evaluator has the necessary background and experience and is thoroughly trained to carry out this important function.

Minimal qualifications for evaluators are as follows: • Team Leaders – • two years of related education or administrative experience; and (b) previous service on site visits with ABHES or other recognized accrediting agencies • Program Specialists – • two years of related experience; (b) Education related to the discipline (c) credentialing in the field, as required for employment, (d) evidence of teaching or development of curriculum or service as a practitioner, and (e) evidence of currency in the field through active participation in the profession. Note: Team leaders used for the evaluation of a program resulting in an academic associate degree or a bachelor’s degree must hold a credential beyond that being evaluated and possess the necessary educational and/or experiential background to effectively evaluate the degree.

New evaluators must submit the following documents prior to serving on a team: 1. A signed statement agreeing to adhere to the ABHES statement of confidentiality, Disclosure and Recusal Concerning Actions Involving Potential Conflicts of Interest. 2. A completed Evaluator General Information, Expertise, & Training Form. 3. A current resume or vitae which must document all educational and experiential background, including the minimum qualifications described above.

Minimal training requirements for new evaluators are as follows: • Experienced evaluators who have not participated on a site visit in three (3) years: • Level 1- • (a) Review of the ABHES Evaluator Training Workshop Manual; and • (b) Submission of Assessment following review. Level 1 – Review of ABHES Evaluator Training Workshop Manual, submission of Assessment following review, and participation in the ABHES Mentorship Program (pairing a seasoned evaluator with a new evaluator, allowing for in-depth conversation, including time for questions and answers, both before, during, and after the evaluation visit). Level 2 –Participation in on-line training (currently in developmental stages) and participation in the ABHES Mentorship Program as described above. Level 3 –Participation in an on-site Subject Specialist or Team Leader Evaluator Training Workshop.

ABHES also offers Team Leader Training Workshops and program-specific evaluator training in certain fields. In all cases, new evaluators will have the ABHES staff at their sides during initial school visits, to answer questions and direct the evaluator as necessary. The team leader will also be a great source of information. Of utmost importance to ABHES is that it provides the necessary information and tools to its evaluators so that they may accomplish the evaluation task effectively and in accordance with ABHES guidelines. Thank you for all you do for ABHES!

Procedures for Visit Arrangements 1. Staff reviews the Self Evaluation Report (SER) and/or other necessary documentation to determine required team composition and completeness of documentation. 2. Staff contacts school to arrange the date for the visit. 3. Staff contacts evaluators to ask availability to serve on visit. 4. Staff confirms hotel and reserves a single room for each team member, to be included in confirmation letter to school and evaluators. **NOTE: Evaluators are responsible for arranging and paying for their own travel, normally air travel, to and from appropriate city. Reimbursement, with appropriate receipts, is promptly paid following the site visit. 5. Following confirmation of visitation team members and hotel, staff provides the confirmation letter for the visit, including the following additional information: • General information on the visitation process • Contact information for visitation team members • Listing of documentation and other items to be available to the visitation team upon arrival • Update Information Form to be completed by the school (updated information not contained in the previous documentation) and mailed to ABHES and each team member prior to the visit • Evaluation Visit Survey to be completed by the school following the visit • Copy of On-Site Student Survey (to be reproduced for the number of students in program(s) being evaluated)

Procedures for Visit Arrangements (continued) Staff copies letter to visitation team members, including the following information: •ABHES Accreditation Manual (if new evaluator), to include any replacement pages if updates have been made • Expense Reimbursement Form, to be completed following the visit • Overview of team members' individual responsibilities • Sample interview questions for interviewing students, faculty, and externship sites • Copy of student survey (to be reproduced at school for number of students in program being evaluated) 6. Staff and evaluators make necessary, individual travel arrangements. 7. The SER comes directly from the school to the ABHES staff member and each visitation team member normally no later than three (3) weeks prior to the visit and the Updated Information Form arrive within 7 business days prior to the visit. 8. The Team Leader contacts each team member by telephone or in writing at least one week prior to the visit to discuss meeting plans and evaluation report assignments. 9. The team meeting is held the evening before or morning just prior to the visit to discuss assignments, outstanding questions or concerns raised following review of the SER, and visitation schedule. 10. Staff drives or arranges for other transportation to the school and the visit begins.

Responsibilities Team Leader Secure transportation to visitation city Review Self-Evaluation Report and other relevant materials, identifying questions or concerns, prior to contacting team or pre-visit team meeting Schedule pre-visit team meeting (with staff) Conduct pre-visit team meeting (See Pre-Visit Team Meeting summary) Control all aspects of visit, including opening session with institution, scheduling of team meetings throughout visit, and ensuring team members efficiently carry out responsibilities. Conduct exit interview with the institution (See Exit Interview summary) Submit own reimbursement forms, including receipts Specialists Secure own transportation to visitation city Review Self-Evaluation Report, identifying questions or concerns relative to program, prior to pre-visit team meeting Research and prepare appropriate sections of visitation report prior to end of visit Submit own reimbursement forms, including receipts

ON-SITE VISITATION PROCEDURES 1. Visitation team meets with Chief Executive Officer of institution and/or other key personnel: • team is shown to their workroom • introductions are made, including brief background of each evaluator • the schedule and outstanding issues are addressed by Team Leader and/or staff • general questions are asked and comments made by visitation team and/or school personnel 2. A tour of the facility is made, including brief introductions to key administrative and teaching staff. 3. Team members begin individual review, including interviewing, class observation, externship visits, and file review. 4. Team breaks for lunch (normally brought in by school) and discusses status of review. 5. Review continues and writing of report commences. 6. If evening school, team normally stays to observe and interview students and all instructors. 7. Following completion of visitation reports, each is reviewed and read aloud among the team members; modifications made, as necessary. 8. The exit interview takes place between the team members and the Chief Executive Officer or other key personnel. The Team Leader identifies the areas of noncompliance and strengths of the program. This is not a time for debate with the institution's representatives over the team's findings but rather an opportunity for the institution to be told the key concerns and begin working on correction prior to receipt of the full visit reports.

Report writing is distributed among the evaluators and staff is assigned a few administrative tasks (e.g. review of catalog). The Team Leader is responsible for the administrative report while the subject specialist is responsible for the program report. Report writing 101 While writing your sections of the report, please keep in mind the following: •Write legibly. The ABHES staff member or, at times, Team Leader will later need to read and interpret what you have written. Complete sentences must be used. •Answers resulting in "Meets Standard" with no request for additional information generally do not need expansion. However, use your own judgment in what is important to the Commission and what they need to know when reviewing the report. •Answers resulting in "Violates Standard" or “Meets Standard” with concerns must include specific, detailed explanations. For example, if documentation is missing from student files each student name should be identified with an explanation of missing documentation. If every student file is reviewed (small school), state this and photocopy examples. ABHES staff can assist team members with the appropriate criteria or standard to be cited. •Only under rare circumstances will exhibits (attachments to the report) be used. Attaching documentation describing how the institution is doing something correctly is not normally needed and may confuse the Commissioners. Should attachments need to be referenced, it is critical a copy of the exhibit be provided to the ABHES staff member. •Areas of non-compliance (violations to standards) must be explained in detail, citing specific examples. This allows the institution to adequately respond to the report. ABHES staff can assist team members with the appropriate criteria and proper documentation. •While each committee member is responsible for writing his or her assigned sections of the report during the visits, the ABHES staff member or Team Leader is responsible for the completion of the final report.

Evaluator Reminders KNOW ABHES STANDARDS. While staff is normally on visits to assist evaluators in locating, following, and understanding ABHES standards, be familiar. Also, don't be specific with regard to other agency standards (e.g., Federal financial aid, OSHA, CLIA, ADA) except in overall statements of concern to assist and warn the institution of potential non-compliance and to protect the ABHES evaluation process. • REMEMBER YOUR ROLE AS AN EVALUATOR. The purpose of the visitation is to confirm what the institution has written in its SER and its compliance with ABHES criteria and standards. Judging the institution based on an evaluator's personal or professional beliefs is not the purpose of the visitation. • AVOID COMPARISON. Comparisons with an evaluator's own school or organization should not be made; either in writing or verbally. Whether comparisons are voiced to the school or to the other team members is not important as such discussion is inappropriate and disruptive to the process. • DO NOT STAY IN THE TEAM WORKROOM ALL DAY. Evaluators are urged to be visible to the institution's administration, faculty and students. While file review and report writing is necessary throughout the visit, it is important to observe and interview frequently as it is by these means that the most relevant information is learned.

Evaluator Reminders (continued) • BE PUNCTUAL. It is important to keep in mind that the staff is responsible for numerous visits, sometimes several in the same week, and must stay on schedule. Your assistance in being prompt to team meetings, meals, and being packed, checked-out and ready to leave the hotel on the last morning helps staff immensely. THINK BEFORE YOU SPEAK. Be aware that comments made during the visitation may come back to haunt the visitation process later. Do not express your opinions to school personnel (e.g., this is a great program!) or discuss intimate details of your own profession (e.g., compared to my teachers, you should be paid more). The staff member and or Team Leader is responsible for relaying information to the institution and only after confirmation of a concern has been made. • WATCH EXPENSES. While ABHES strives for evaluators to be comfortable as evidenced through hotel chains used and meals made available, keep in mind that every penny spent on this visit will be reimbursed by the school. For large teams, small schools, and schools facing financially difficult times (as most are now), this means quite a bite into budgets. When choices are available, choose the less expensive of the two. Traveling for ABHES as you travel personally is always a good way to conserve money.

Types of visits New (initial) Refers to a full evaluation visit to an institution or program seeking new (initial) accreditation from ABHES. This visit is conducted following receipt of all required documentation, including the completed Self Evaluation Report (SER). The evaluation team consists of a Team Leader, staff member, and subject (program) specialist(s). Minimally, the evaluation team consists of 3 individuals, including the ABHES staff person. Report Used: Visitation Committee Reports (Administrative and Subject Specialist) Recycle (renewal) Refers to a full evaluation visit to an institution or program seeking renewed accreditation from ABHES. The application and SER materials are sent by ABHES to currently-accredited institutions and programs approximately 18 months prior to the expiration of the institution's grant of accreditation. ABHES staff determine which visitations will be conducted during the first half of the year and which will be conducted in the last half of the year, thus requiring two different SER submission dates. The evaluation visit is normally conducted within 6 months of receipt of all required documentation, including the completed SER. The evaluation team consists of a Team Leader, subject (program) specialist(s), and ABHES staff member. Minimally, the team consists of 3 individuals, including the staff person. Report Used: Visitation Committee Reports (Administrative and Subject Specialist) *NOTE: Please visit www.abhes.org, under “Evaluator” tab to locate reports.

Communication During and After the Visit Discuss areas of concern with the team leader and the staff member as they arise. Share what you have found with the rest of the team first and NEVER with institutional personnel or students initially. Institutions must be given the opportunity to address potential areas of noncompliance before the exit interview and this opportunity will be provided through the team leader and/or staff. An evaluator’s only contact with an institution should be during the site visit. Communications between an institution or program representative and an evaluator prior to or after a site visit are inappropriate. Move outside of the meeting room frequently and talk with staff, faculty, and students. While the comments received are very important, a violation to standards cannot be cited unless documented evidence backs up any claims. Be discreet when talking with staff, faculty, students and graduates. There are issues of both privacy and confidentiality at stake.

Rules of the Road- Accepting a Visit Assignment ABHES assigns evaluators based upon areas of expertise or specialty, level of credential conferred by the program being evaluated, area of expertise on team (e.g., specialist, team leader), geography, and conflict of interest considerations. If a visit to a particular institution may be a potential conflict of interest, make this known as soon as you become aware of the problem. If you have visited the institution or program before, please notify the staff. We’d like to “broadcast” your talents by not sending you to the same place twice if possible. You are the best one to determine whether you are a good fit for the visit, so advise the staff coordinating the visit if you have any concerns. Travel Plan to stay for the entire visit, per the direction of the coordinating staff, and do not plan departing flight too early. Be considerate in purchasing your airfare in a timely manner.

Rules of the Road- (continued) Expenses Consider the following: Your expenses are ultimately paid by someone else. Exercise consideration and moderation when and where possible. You are initially responsible for your expenses, using a personal credit card, and you will be reimbursed for expenses expeditiously following receipt of a completed Expense Reimbursement Form, including receipts. Your preferred airline might not be the one you fly most frequently (i.e., the one that will generate the most “mileage”). Instead, we ask that you consider carefully the airline that will make the visit more economical while still convenient to you. Book air travel early and save on expense. Only seek reimbursement for yourself. Seek prior approval from the staff member coordinating the visit for: Airline tickets exceeding $600, rental vehicle, extra-night hotel stays.

Rules of the Road- (continued) ABHES will not reimburse for such things as: Costs for entertainment, including theatre plays, cinema, athletic events, or amusement parks, even if submitted in lieu of meals. In-room entertainment. Excessive use of hotel room telephone. Alcohol purchase must be reasonable in both cost and consumption. **Review the Expense Reimbursement Form for additional information on expenses. Your Way is Not Always the Only Way Remember that you are representing ABHES when visiting an institution or program and not representing your own organization. The question isn’t whether this institution or program “does it the way yours does;” but instead whether the institution or program provides its educational product in a reasonable and suitable manner in light of its mission, size, location and the communities served and taking into consideration ABHES’ accreditation standards. Ultimately, this is a peer review process. Do not judge more harshly than you would be willing to be judged yourself and never beyond the ABHES accreditation standards. Gifts or Services Do not ask for or take gifts or services from institutions or programs you are visiting. Such offers, though sincere, should not be accepted in order to avoid any possible misconceptions or the appearance of impropriety. Do not discuss or accept a consulting or employment position with an institution you are evaluating (at least until after ABHES has finally resolved the institution’s pending application).

After the Visit 1. Staff finalizes and consolidates visitation team reports, copies and submits the report to the school, including directions and timeline, for response. 2. Requests for expense reimbursements from evaluators are received by staff and reimbursed in a timely manner (normally checks are cut weekly). 3. A response from the institution is received, normally within 3 weeks of receipt of the team report. 4. The Preliminary Review Committee reviews all information relative to the school being reviewed and completes a File Review Sheet, including a recommendation for action. 5. All information on the school, including the Self-Evaluation, Team Report, institution's response, and File Review Sheet are both hard copied and electronically supplied to the Commission for review prior to the scheduled meeting. 6. Commission meets and takes action. 7. ABHES Executive Director advises institution in writing of Commission action within three weeks of the meeting.

Forms and Reports Please visit our website, www.abhes.org, under the ‘Evaluator’ tab, you can find all of the documents you will need. Expense form Evaluator Responsibilities On-site Visitation Survey Visit Reports Institutional or Programmatic

And Finally… Remain friendly. Be professional. Be ethical. Stay cool. None of this is personal. Enjoy the visit! It can be an exceedingly rewarding and educational experience for all involved.

Congratulations! You have just completed the first portion of the Level 1Evaluator Training! What’s the next step? You must complete the assessment. Email evaluator@abhes.org for the assessment link.

Importance of Evaluator Training in ABHES Accreditation Process