230 likes | 355 Views
Self-Assessment of Surgical Skill Acquisition with Computer-Based Video Training and the Impact on Self-Directed Training Nathan Jowett Vicki LeBlanc George Xeroulis Helen MacRae Adam Dubrowski Surgical Skills Centre The Wilson Centre University of Toronto, Toronto, Ontario, Canada.
E N D
Self-Assessment of Surgical Skill Acquisition with Computer-Based Video Training and the Impact on Self-Directed Training Nathan Jowett Vicki LeBlanc George Xeroulis Helen MacRae Adam Dubrowski Surgical Skills Centre The Wilson Centre University of Toronto, Toronto, Ontario, Canada
Is CBVT as effective as expert feedback in basic surgical skill acquisition? Rogers et al (1998): CBVT vs. didactic lecture Equal cognitive performance but lower technical performance for CBVT group Summers et al (1999): CBVT with expert vs. didactic instruction CBVT superior on performance retention testing Rogers et al (2000): CBVT vs. Concurrent feedback with CBVT Improved performance by external feedback group Xeroulis et al (2005): CBVT vs. expert feedback No difference in performance between groups
Research Objective Are trainees able to adequately acquire basic technical skills on their own practice schedules using CBVT? Research Question: Is technical skill acquisition by means of CBVT compromised when trainees practice in unstructured as opposed to structured learning settings? Hypothesis: Yes
Experimental Paradigm Structured Practice (SP) N = 30 Mean open practice time = 22.7 minutes Post-Test 1 Post-Test 2 Retention Test Pre-Test Self-assessed proficiency (1 week delay) Unstructured Practice (UP) Video Captured Trials with Hand Motion Analysis Self-assessment GRS + Proficiency, Expert GRS Self-assessment GRS + Proficiency Expert GRS Practice block (3 minutes) Open practice period (variable) Additional practice period (fixed)
Learning Curve: Time 350 Structured Unstructured 300 250 200 Total time (s) 150 100 50 0 1 2 3 4 0 25 50 75 100 Retention Normalized time to self-assessed proficiency (%) Trials after proficiency
Time Practice Group Structured Unstructured Time (s) Pre-test Post-test 1 Retention (Self-assessed proficiency)
Hand Movements Practice Group Structured Unstructured Number of Hand Movements Pre-test Post-test 1 Retention (Self-assessed proficiency)
Path Length Practice Group Structured Unstructured Path Length (decimetres) Pre-test Post-test 1 Retention (Self-assessed proficiency)
Expert Global Rating Scale Practice Group Structured Unstructured Expert Global Rating Scale Pre-test Post-test 1 Retention (Self-assessed proficiency)
Summary Did significant performance improvement occur between pre and retention tests for both groups? Yes (p < 0.05) Did SP and UP groups differ on retention test performance? No (p > 0.05) Did improvement occur between first and second post-tests among SP participants? No (p > 0.05)
Conclusions Trainees learning basic knot tying skills using CBVT in an unstructured setting performed equally as well as those in a structured setting This suggests that trainees were able to accurately self-assess the progression of their technical performance and thus determine when further practice in a single session would no longer be beneficial
Implications Our results support the initiation of self-directed online or CD-ROM CBVT for basic technical skills as a plausible solution to the economic and staffing constraints associated with teaching technical skills outside of the O.R.
Supported by: Dean’s Excellence Fund Faculty of Medicine, University of Toronto Medical Student Summer Scholarship Faculty of Medicine, University of Toronto
exact) Randomization Efficacy Pre - intervention variable Structured Unstructured P Value* Years of medical school training, mean 1.40 (0.70) 1.55 (0.51) 0.51 Sex, No (%) Female 6 (60) 7 (35) 0.26 Male 4 (40) 13 (65) Experience: two - handed square knot, No (%) Yes 1 (10) 1 (5) 1.00 No 9 (90) 19 (95) Experience: instrument tie, No (%) Yes 6 (60) 10 (50) 0.71 No 4 (40) 10 (50) General self efficacy score†, mean (SD) 4.18 (0.46) 3.97 (0.39) 0.21 Practice time to self (min), (SD) - assessed proficiency 18.30 (6.40) 24.90 (13.67) 0.16 2 *Independent t test for continuous data (2 - tailed), Pearson χ test for nominal data (2 - tailed, †Sherer general self - efficacy scale
Post tests 1 and 2 comparisons (SP group) Table 6. Paired-samples t test for performance variables between first and second post tests (structured practice group).
Mean Performance: Pre, Post-1, Retention Performance variable Pre-test Post-test Retention-test Mean global rating scale score (/20) 4.5 11.6* 10.4* Structured Unstructured Average P>.05 4.5 11.0* 8.7* 4.5 11.2* 9.2* Hand-motion analysis Time (seconds) 158.5 36.7* 37.5* Structured Unstructured Average P>.05 183.0 41.9* 62.2* 174.6 40.2* 54.0* Number of hand movements Structured Unstructured Average 96.6 35.3* 37.3* P>.05 135.8 38.5* 49.3* 122.3 37.4* 45.4*