Objectives for In-Training Exam

1. ASCO In-Training ExamFor more information go to Collichio et al, JCO 2009Kenneth Hande, MDVanderbilt UniversityChair, Test Materials Development Committee

2. Objectives for In-Training Exam Stimulate learning Provide as assessment for fellows as to how they are progressing compared to other fellows in training Provide as assessment to Program Directors as to how successful their training program is in educating their fellows compared to other programs Give fellows an idea of what the ABIM certification exam will be like (however, the In-Training Exam is not a duplicate of the ABIM certification exam) Give assistance to Program Directors to assist receiving ACGME approval.

3. Timeline for In-Training Exam December: volunteers recruited to write questions for exam Experts in various areas of oncology Have participated in a National Board of Medical Examiners (NBME) question writing workshop Current members: Bay State, City of Hope, Columbia, Memorial, Michigan, Montefiore, National Cancer Institute, North Carolina, Palo Alto, Stanford, Vanderbilt, Vermont January-April: committee writes questions in their assigned area May-July: NBME staff edits and reviews questions July: Committee meets to review all questions and grades submitted questions

4. Timeline for In-Training Exam July-October: NBME reviews questions to fill various sections of test (basic science, clinical trial design, GI, GU etc). Highest graded questions are selected. November: Best questions from this year (2/3) and best questions from previous years (1/3) are presented to 3 person chairman’s subcommittee. Final 200 questions are selected February: Test administered 200 questions Web-based 6 hours March: Questions that NBME flag are reviewed by the chairman’s committee for accuracy. Inaccurate questions are omitted from scoring. April: Scores are returned to fellows and program directors

5. In-Service Exam Question Breakdown Ethical/Competency/supportive care/palliative – 12% Basic science (pharm, trial design, statistic, biology, tumor markers) – 15% Hematologic malignancies – 15% Breast – 12% GI – 11% Thoracic – 8% GU – 8% Gyn – 7% Other cancers – 12%

6. Overall Statistics

7. 2008-2009 Registration Breakdown

8. Score Report Contents • Program Director Report • Compares programs fellows by PGY to aggregate in all content areas • Program Roster • Provides total test score and all content area scores for each examinee • Program Interpretation Guidelines • Provides mean score for each content area by PGY as well as providing percentile ranks by scaled score and PGY • Program Histogram • Compares programs mean score to mean scores of all other programs • Examinee Score Sheet • Displays performance bands for each content area compared to a base reference group

9. Histogram: All Examinees per Program 2008 2009

10. Fellow’s Score Report • CONTENT AREAS: • - Competencies • - Basic Science Principles • - Hematologic Malignancies • - Breast Cancer • - GI Cancer • - Lung Cancer • - GI Cancer • - Gynecologic Cancer • Other Cancers • ADDITIONAL AREAS: • Diagnosis • Patient Management • Prognosis

11. Program Interpretation Guidelines

12. In-Training Exam Feedback • 86% answered survey • 79% indicated appropriate # of questions • 78% indicated questions were clearly written (6.8% said they were not clearly written) • 71% said exam was moderately difficult • 71% said it was a very positive or positive experience. 1.6% said it was a negative experience.