1 / 11

The Objective Structured Assessment of Technical Skills (OSATS)

The Objective Structured Assessment of Technical Skills (OSATS). Helen M. MacRae, M.D., FRCSC, D.H. Gales Director Technical Skills Centre, University of Toronto. Certifying competence of new trainees. The Need for a Reliable and Valid Measure. Certifying Competence

abrianna
Download Presentation

The Objective Structured Assessment of Technical Skills (OSATS)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Objective Structured Assessment of Technical Skills (OSATS) Helen M. MacRae, M.D., FRCSC, D.H. Gales Director Technical Skills Centre, University of Toronto

  2. Certifying competence of new trainees The Need for a Reliable and Valid Measure Certifying Competence in New Technology (for example laparoscopic surgery or robotic surgery) Assessing the impact of interventions

  3. The OSATS • A multi-station “bell-ringer” examination • Segments of 6 to 8 procedures • Performed on “bench models” simulating relevant anatomical structures • Knowledgeable OR nurses assisting

  4. The OSATS (continued) • 15 minutes per procedure • Candidates move from station to station • Each station marked by qualified surgeon • Examiner uses two marking systems • Task-specific checklists • Global rating scales

  5. Sample of Checklist Scoring (5 of 24) Vascular Anastamosis Station 0 1 • Selects appropriate suture • Avoids excessive trauma / handling of artery with instruments • Guides suture down to desired position following bites • Uses correct needle angle >80% time. • Follows curve of needle on entrance / exit > 80% time 1 0 1 0 0 1 0 1

  6. Sample of Global Rating (3 of 5) Vascular Anastamosis Station Respect for tissue 1 2 3 4 5 Frequently caused damage by inappropriate use of instruments Careful handling of tissue but occasionally caused inadvertent damage Consistently handled tissue appropriately with minimal damage to tissue Time and motion 1 2 3 4 5 Many unnecessary moves Efficient time/motion but some unnecessary moves Clear economy of movement and maximum efficiency Flow of Operation 1 2 3 4 5 Frequently stopped operating and seemed unsure of next move Demonstrated some forward planning with reasonable progression of procedure Obviously planned course of operation with effortless flow from one move to the next

  7. OSATS - Reliability • Internal consistency (across stations) • Inter-rater reliability (within stations)

  8. Validity Measures • Clinical PGY level • are more senior trainees scoring higher? • Live animal version of exam • are better “live model” operators scoring higher? • Evaluation of final product • are those with better outcomes scoring higher? • Faculty rankings of clinical performance • are better ranked trainees scoring higher?

  9. OSATS - Validity • Correlation with PGY level • Correlation with other measures

  10. Summary of OSATS • A non-patient lab-based platform for testing technical skills which shows: • Good Reliability • Good Validity • Widespread Feasibility • Widespread Applicability

  11. Conclusions We can reliably and validly test some of the procedural skills of medicine in a formal, structured manner outside of the clinical setting

More Related