Scoring the Clinical Proficiency Examination

1. Scoring the Clinical Proficiency Examination Dr. Beth Sabin, Assistant Director AVMA Education and Research Division CLEAR Annual Conference September 16, 2005

2. Clinical Proficiency Examination • 3.5-day, 7-section, hands-on assessment of clinical veterinary medical skills • Must pass all sections • Administered at multiple ECFVG-approved sites according to Manual of Administration • Also administered by the Canadian National Examining Board (NEB)

3. Educational Commission for Foreign Veterinary Graduates • Seven-member AVMA committee • Oversees educational equivalency assessment program • Candidates are graduates of international non–AVMA-accredited veterinary schools • Certificate holders meet educational prerequisite for licensure or employment

4. ECFVG Program Steps • Enroll; proof of graduation • Assessment of English language competency (ETS, CAEL, IELTS) • Assessment of basic and clinical sciences knowledge (NAVLE) • Assessment of hands on clinical skills (CPE, postgraduate ECY)

5. Clinical Proficiency Examination • 3.5-day, 7-section, hands-on assessment of clinical veterinary medical skills • Anesthesiology • Clinical and Anatomic Pathology • Equine Practice • Food Animal Practice • Radiology • Small Animal Practice • Surgery

6. Clinical Proficiency Skills CPE Manual of Administration defines as: “…combination of knowledge and technical skills.” Examiner assesses knowledge by directly observing examinee behaviors as examinee performs basic skills required of a minimally competent entry-level practitioner.

7. CPE Audit • Audit conducted in Spring 2001 • Findings provided to ECFVG Fall 2001 • CPE accomplishes the intended goal - it assesses the minimal clinical competency of a graduate of a non-accredited veterinary program • Suggestions for enhancements

8. Post-audit CPE Activities • Examiners’ Training • Review of CPE Manual of Administration • Examiners’ Master Manual • Review of CPE scoring parameters • Survey of faculty and students

9. CPE History • Experts selected most pertinent skills to evaluate performance competency • Scoring process defined • Goal is to separate “masters” from “non-masters” • Interested in evaluating for minimal competency

10. Performance Benchmarks • Consider the behaviors to be observed and what constitutes minimal levels of performance at each CPE scoring level: • Superior • Good • Satisfactory • Unsatisfactory

11. CPE Rating Scale(for six of seven sections) 100% - 86% Superior Superior performance of required clinical skills 85% - 71% Good Good performance of required clinical skills 70% - 61% Satisfactory Satisfactory performance of required clinical skills < 60% Unsatisfactory Failed to demonstrate the required clinical skills

12. Development of Assessment Methods • Competencies identified for each section • Example—Clinical and Anatomic Pathology • Recognize and interpret visual images of gross pathological lesions • Perform a necropsy and take tissue samples for histopathology • Perform common laboratory tests and interpret results of those tests in the areas of clinical microbiology, clinical parasitology, hematology, and urinalysis

13. Assessment of Skills • Skills/activities identified for each competency • Example—Perform necropsy • Thoroughly examine intact carcass • Opens carcass, examine major cavities, remove viscera and head • Examine major organs • Examine muscles, joints, bone marrow • Examine endocrine glands, lymph nodes

14. Behavioral Anchors • Observable behaviors identified for each skill/activity • Example—Thoroughly examine intact carcass (5 pts) • 1 pt: Assesses general body condition/hydration • 1 pt: Examines skin/hair coat • 1 pt: Examines head • 1 pt: Examines limbs including feet • 1 pt: Examines perianal and genital region, mammary glands

15. Calculating Section Scores • Determine weighting of each competency • Example—Clinical and Anatomic Pathology • Recognize and interpret visual images: 16% • Perform necropsy: 20% • Perform lab tests and interpret results: 64% • 16% for each of four areas • Sum weighted station scores = section score • Pass = 60

16. CPE Rating Scale(for surgery) • 2004 review by subject matter experts and testing consultants • Minimal competency best assessed via pass/fail • Experts developed: • Check list of passing behaviors • Check list of failing behaviors • Appendix of fatal flaws

17. Scoring Surgery CPE Manual states: • A “pass” = at least a minimally competent performance of entry-level surgical skills • Examiner to use the surgical skills assessment sheet as a checklist and to record positive and negative behaviors • Only feedback provided is in the case of a fatal flaw

18. Surgery Competencies • Prepare the patient • Prepare self • Perform surgical procedure

19. Surgical Skills • Asepsis • Ovariohysterectomy technique • Tissue trauma • Hemostasis

20. Observable Behaviors Tissue trauma • Passing behaviors • Demonstrates atraumatic tissue handling throughout • Skills do not result in cumulative traumatic handling of tissue • Failing behaviors • Causes significant damage to other tissues/organs (fatal) • Skills result in cumulative traumatic handling of tissue

21. Failing Surgery CPE Manual states: • Should a candidate exhibit any one behavior in the “Failing Behaviors” column of the assessment sheet, he/she will be allowed to complete the Surgery section but will receive a failing grade

22. Fatal Flaws CPE Manual States: • Should the candidate commit a fatal flaw, the examiner will interrupt the examination, correct the error to ensure safety of the animal and personnel, and award the candidate a failing grade. • The candidate will not be allowed to complete the Surgery section of the CPE.

23. Five Fatal Flaws • Fails to control significant hemorrhage • Fails to create a secure closure of the abdominal wall • Unable to achieve and maintain an aseptic field • Causes significant damage to other tissues/organs • Exhibits any other behavior that puts the animal’s life at risk

24. Radiology • Ongoing review by subject matter experts • Development of consensus on types of radiographs and radiographic lesions • Asking experts: “What types of lesions should a minimally competent entry-level veterinarian be able to identify and in what species?”

25. Goals of Radiology Review • Long-term • Develop a central bank of radiographs • Short-term • Update competencies, skills/activities, and observable behaviors for assessment of minimal entry-level competency in radiology

26. Additional Information • ECFVG Web site • www.avma.org/education/ecfvg/default.asp • Contact ECFVG office • ECFVG@avma.org • 800-248-2862, ext 6675