Download
1 / 25
250 likes | 394 Views
Clinical Skills Verification Rater Training MODULE 4 Strategies for Clinical Skills Assessment: Models and Best Practices. Michael Jibson , M.D., Ph.D. David Kaye , M.D. Sandra Sexson , M.D. Module 4 Pre-Test.
E N D
Clinical Skills Verification Rater TrainingMODULE 4Strategies for Clinical Skills Assessment:Models and Best Practices Michael Jibson, M.D., Ph.D. David Kaye , M.D. Sandra Sexson , M.D.
Module 4 Pre-Test • The strength of a Clinical Skills Verification Examination stems from its ability to serve as a forum for all of the following except: A. A high stakes examination B. direct observation of clinical skills C. provision of immediate feedback on performance D. repeated administrations during residency
Module 4 Pre-Test • Which of the following models for CSV exam is the least standardized? A.Workshop Model B. Mock Board Model C. Clinical Evaluation Model
Module 4 Pre-Test 3. Which of the following models for the CSV exam makes the greatest use of resident peer feedback? A.Workshop Model B. Mock Board Model C. Clinical Evaluation Model
Module 4 Pre-Test 4. Within a program, consistency of faculty expectations and rating of residents on the CSV exam can be maximized by: A. All raters using the same evaluation model B. Limiting assessment to the three competency areas outlined by the ABPN C. Using standardized patients D. Monitoring inter-rater reliability
Clinical Skill Verification (CSV) Strengths of Clinical Skills Verification • Direct observation of clinical skill • Multiple observations and observers • Defined areas for evaluation • Specific anchors for performance • Immediate feedback • Ample opportunity for remediation
Evaluation Model We recommend that the evaluation: • Be done early in residency and repeated frequently • Include immediate feedback • Avoid the “high-stakes exam” model
Evaluation Model Mock Board Model • Use the CSV for the current practice of “Mock Board” exams • Prearranged patients • Scheduled interviews
Advantages Familiar format Highly standardized Disadvantages Retains high-stakes exam format Narrow patient selection Difficult to schedule Mock Board Model
Evaluation Model Workshop Model • Use the CSV form for a workshop or class on interviewing • Involve peer resident observers • Videotape the encounter
Advantages Multiple residents involved Highly standardized Excellent feedback (if videotaped) Disadvantages Few opportunities for each resident to interview Narrow patient selection Workshop Model
Evaluation Model Clinical Model • Use the CSV form for any initial clinical encounter with supervising faculty • Inpatient admission • Emergency room evaluation • Outpatient assessment
Advantages Broad range of patients Many opportunities to interview Easy to schedule Disadvantages Moderately standardized Faculty buy-in essential Clinical Model
Evaluation Model We recommend that the evaluation: • Occur during regularly scheduled clinical rotations • Involve the standard interview used in that clinical setting • Involve randomly selected patients • Be done as part of the routines of clinical care
Additional Issues Programs may want to consider: • Using the exam to test additional competencies for their own purposes • Requiring residents to pass the examination before advancing through training
Additional Competencies CSV requirements may be combined with other clinical competencies: • Case formulation • Diagnostic assessment • Treatment planning
Faculty Training We recommend that programs: • Offer training to faculty in how to conduct and score the examination • Make an effort to standardize scoring on the examination • Monitor inter-rater reliability internally • Consider credentialing faculty as evaluators
Post-Test Module 4 • The strength of a Clinical Skills Verification Examination stems from its ability to serve as a forum for all of the following except: A. A high stakes examination B. direct observation of clinical skills C. provision of immediate feedback on performance D. repeated administrations during residency
Post-Test Module 4 • The strength of a Clinical Skills Verification Examination stems from its ability to serve as a forum for all of the following except: A. A high stakes examination The in –residency CSV exam allows programs to administer repeatedly and avoid the high-stakes aspect.
Post-Test Module 4 • Which of the following models for CSV exam is the least standardized? A.Workshop Model B. Mock Board Model C. Clinical Evaluation Model
Post-Test Module 4 • Which of the following models for CSV exam is the least standardized? C. Clinical Model The clinical model utilizes patients seen on regular clinical services without pre-selection of patients
Post-Test Module 4 3. Which of the following models for the CSV exam makes the greatest use of resident peer feedback? A. Workshop Model B. Mock Board Model C. Clinical Evaluation Model
Post-Test Module 4 • Which of the following models for the CSV exam makes the greatest use of resident peer feedback? A. Workshop Model
Post-Test Module 4 4. Within a program, consistency of faculty expectations and rating of residents on the CSV exam can be maximized by: A. All raters using the same evaluation model B. Limiting assessment to the three competency areas outlined by the ABPN C. Using standardized patients D. Monitoring inter-rater reliability
Post-Test Module 4 • Within a program, consistency of faculty expectations and rating of residents on the CSV exam can be maximized by: D. Monitoring inter-rater reliability Along with faculty training on how to conduct and score the examination, tracking inter-rater reliability among faculty after observing the same performance and giving feedback to faculty regarding their ratings compared to others is a very effective way of standardizing ratings.
