ABA/ASA JOINT COUNCIL ON IN-TRAINING EXAMINATIONS

1. ABA/ASA JOINT COUNCIL ON IN-TRAINING EXAMINATIONS 2004 REPORT TO SAAC/AAPD Glenn P. Gravlee, MD Chair, Joint Council The Ohio State University Medical Center Department of Anesthesiology

2. ABA David Chestnut Douglas Coursin Glenn Gravlee Patricia Kapur Mark Rockoff (Jim DiNardo) Raymond Roy Kenneth Tuman ASA Arnold Berry John Cooper (John Rowlingson) Jeffrey Gross (V.C.) Philip Lebowitz Charles Otto Patricia Petrozza Mark Rosen JOINT COUNCIL MEMBERS

3. 50 Question Writers Title: Junior Editors

4. IN-TRAINING EXAMINATION EDITORS Steven Allen, Audree Bendo, James DiNardo, Carter Dodge, Sylvia Dolinski, John Ebert, John Emhardt, Robert Gaiser, Eric Kitain, Bruce Kleinman, Elliott Krane, Larry Krenis, Larry Kushins, Cynthia Lien, Vinod Malhotra, Donald Martin, Roger Mecca, John Moyers, Julia Pollock, Lindo Jo Rice, Robert Sladen, Richard Stypula, Richard Teplick, Helen Westman, Thomas Wolfe

5. IN-TRAINING EXAMINATION • “3 exams in 1” • In-Training Exam • Requalifying Exam • ABA Written Exam • Subset of 300 questions from ITE • Plus: Recertification Exam • Well-performing A-types from ITE form the question pool

6. Demographics of In-Training Examinations Total Number of Examinees Registrants

7. DEMOGRAPHICS OF IN-TRAINING EXAMINATIONS - AMG + IMG

8. Exam Content340 Questions ~ 100 are Links 15% 16% 4% 8% 9% 47% Number of Questions

9. Disease States Breakdown163 Questions Number of Questions 7.2% 5.7% 5.7% 5.1% 4.9% 4.6% 4.6% 4.6% 4.3%

10. Item Difficulty (Mean Logit ± SE)

11. In-training ExamAverage Scaled Score All CA3 Residents

12. Gaps in Knowledge – CA3s • 8 Surprising Gaps Identified • Report being sent by ASA office

13. Examination Registration/Behavior • Government Issue Picture ID • NO Cell Phones or Pagers • Calculators are NOT OK • Consequences of Cheating • Proctors are in charge

14. Joint Council Finances • Joint Oversight by ASA/ABA • Largely sustained by Exam fees • In the Red ($50-100K) when N was low

15. Joint Council Finances • Still losing $20-50K/year with strong N: increased proctor compensation, NBME fees mainly • Exam cost @ $85 since 1996 • Joint Council approved increase in exam fee to $100 effective for 2005 exam

16. Content Outline Revision • 1996 version used for 2004 Exam • 2003 version used for 2005 Exam • New Format: “Organocentric” • Both versions on ASA website • 2004 version disappears 12/31/04 • www.asahq.org/publicationsandservices/contentoutlinerev2003

17. FAQs/Future Issues • Computerized Exam? • Phase out K-types? • Introduce R-types and question clusters (G types) in 2005

18. Q. Which of the following foods grows on trees? • Bananas • Tomatoes • Apples • Freedom (French) Fries A:1,2,3 B:1,3 C:2,4 D:4 E:1,2,3,4 Answer: B

19. K-type evolution • Always on exam: approx 40% in 1970s and 1980s • Gradual decrease to current 20% • Elimination being considered: primary rationale for evaluating R-types • NBME dislikes, controversial among Joint Council members

20. Disadvantages to K-types • No longer on USMLE • Take longer to answer (approx 60 vs 50 sec) • Slightly more difficult than A-types • Examinees dislike them

21. Advantages to K types • Question performance has been good • Discriminates stronger vs weaker examinees pretty well, ideal K-type requires greater depth of understanding • Anesthesiology doesn’t always lend itself to single best answer • May take 3-4 A-types to test the same information as a single well-written K-type

22. K-type (interim) conclusions • See how R-types perform • Hold the line at 15-20% for now, reassess over next two years • Let’s base K-type fate more upon the need for knowledgeable anesthesiologists than upon question popularity among examinees • Remember that they have 3-4 years to practice

23. Acute DIC Hemophilia A Platelet function defect Idiopathic thrombocytopenia purpura von Willebrand’s disease Antithrombin III deficiency Extended Matching (R-type) 1. 70 year-old man undergoing CABG/AVR is oozing after CPB • A 70 year-old man undergoing CABG/AVR • develops a coagulopathy after CPB. Answer: C

24. Extended Matching (R-type) • Acute DIC • Hemophilia A • Platelet function defect • Idiopathic thrombocytopenia purpura • von Willebrand’s disease • Antithrombin III deficiency An active 30 year-old man with a history of hemarthroses and excessive bleeding after small cuts presents for appendectomy. Answer: B [Continue with additional questions]

25. R-types and G-types Examples soon will appear on ASA Web site

26. Nonstandard Examinations • Extended time, quiet room, reader, etc. • Application processing takes more time (4 months before exam) • Process described in ABA Booklet of Information, Section 7.01 • Same process for ITE as ABA written exam • Once approved by ITE, ABA usually rubber-stamps it (same committee)

27. Nonstandard Exam: Concerns • Numbers gradually increasing • Candidates often wait for ABA exam to request accommodation • Candidates/examinees often allow insufficient time • Candidates/examinees presume that a previous accommodation automatically merits an ABA/ASA or ABA accommodation

28. Nonstandard Exam Concerns • Recognize that A.D.D. does not automatically qualify for extended test time • Recognize that psychological testing must be <5 years old • Comments from candidates and from our expert consultants at times scare the Nonstandard Exams Committee

29. “When a resident fails to measure up to the academic standards and/or professional behavior expected, it is incumbent upon the program director to document these failures during the Residency Review Committee-mandated semiannual review.” Miller SH, Plastic Reconst Surg 1990

30. Communications gravlee.1@osu.edu 614-293-9081 Therese O’Donnell-Leonard