Task Deconstruction and Metrics

1. Task Deconstruction and Metrics Robert M. Sweet, MD Assistant Professor Urologic Surgery and General Surgery Clinical Director of Simulation Programs Founding Director, CREST Robert Sweet, MD

2. Procedure vs. Task vs. Exercise • Procedure-is a “series of steps taken to accomplish an end” • Task-“piece of work to be done” • Exercise-”effort performed to develop or maintain fitness or increase skill” Webster’s Dictionary Robert Sweet, MD

3. Task deconstruction • Procedures can be broken down into logical components • Guidelines for the “order by which this occurs” • Safety considerations • Natural progression of skills from simple to complex • Resources and expense • Independent opportunities for learning loaded on the front end, but sprinkled throughout Robert Sweet, MD

4. Learning Domains in Simulation Training Cognitive + Technical + Communication Performance = Dependent Psychomotor Situational Awareness Closed-Loop Communication Independent Visual-spatial Leadership Transfer Team Formation and Reformation Automatic Robert Sweet, MD

5. Classify your learners • Who are your learners • Where are they as far as their level of understanding? Robert Sweet, MD

6. Types of Knowledge • Declarative • Knowing “what” • Procedural • Knowing “how” • Schematic • Knowing “why” • Strategic • Knowing how and when to apply Wilson et al, Systems for State Science Assessment, 2005 Robert Sweet, MD

7. “Discriminate the differences between IJ and Subclavian central lines” “What would you do if . . . . .” “Asking for the proper equipment you need to have available to put in a central line” “Name the venous branches leading to the SVC” Learning Domains-Cognitive Bloom’s Taxonomy Robert Sweet, MD

8. Robert Sweet, MD

9. Cognitive Domain Tasks • Usually good to load on the front-end of the curriculum as it meets the criteria • Safe • Low resources • Logical to understand before doing • Estimated to comprise 75% of what we learn in procedures (Spencer et al.) Robert Sweet, MD

10. Teaching and assessing judgment Red Llama, Inc.:SimPraxis Robert Sweet, MD

11. Situational Awareness Closed loop communication Leadership Transfer Team Formation and Reformation Communication Domain Robert Sweet, MD

12. Psychomotor/Visual-spatial Domain Naturalization Articulation Precision Manipulation Imitation Dave, R. H. (1975). Developing and Writing Behavioural Objectives. (R J Armstrong, ed.) Educational Innovators Press Robert Sweet, MD Dave’s Taxonomy

13. Psychomotor/Visual-spatial Domain Naturalization Articulation Precision Manipulation Imitation Observing and patterning behavior after someone else Dave, R. H. (1975). Developing and Writing Behavioural Objectives. (R J Armstrong, ed.) Educational Innovators Press Robert Sweet, MD Dave’s Taxonomy

14. Psychomotor/Visual-spatial Domain Naturalization Articulation Precision Manipulation Performance after training, with errors Imitation Observing and patterning behavior after someone else Dave, R. H. (1975). Developing and Writing Behavioural Objectives. (R J Armstrong, ed.) Educational Innovators Press Robert Sweet, MD Dave’s Taxonomy

15. Psychomotor/Visual-spatial Domain Naturalization Articulation Precision Refining technique with few errors Manipulation Performance after training, with errors Imitation Observing and patterning behavior after someone else Dave, R. H. (1975). Developing and Writing Behavioural Objectives. (R J Armstrong, ed.) Educational Innovators Press Robert Sweet, MD Dave’s Taxonomy

16. Psychomotor/Visual-spatial Domain Naturalization Articulation Multi-tasking Precision Refining technique with few errors Manipulation Performance after training, with errors Imitation Observing and patterning behavior after someone else Dave, R. H. (1975). Developing and Writing Behavioural Objectives. (R J Armstrong, ed.) Educational Innovators Press Robert Sweet, MD Dave’s Taxonomy

17. Psychomotor/Visual-spatial Domain Naturalization High-level of performance on auto-pilot Articulation Multi-tasking Precision Refining technique with few errors Manipulation Performance after training, with errors Imitation Observing and patterning behavior after someone else Dave, R. H. (1975). Developing and Writing Behavioural Objectives. (R J Armstrong, ed.) Educational Innovators Press Robert Sweet, MD Dave’s Taxonomy

18. Psychomotor/Visual-Spatial Skills • Basic skills (adapted from Satava et al., 2002) • Patient positioning • Knowing steps of procedure • Knowing equipment needed • Sterilization procedures • Instrument handling • Bimanual Dexterity • Navigation • Suturing • Knot-tying • Intubation • Palpation • Cannulation • Cutting • Incising • Exploration • Tissue handling Berkley et al.,2000 Robert Sweet, MD

19. Psychomotor/Visual-Spatial Skills • Intermediate Skills (adapted from Satava et al., 2002) • Complex wound closures • Hemorrhage control • Endoscopy • Excision • Anastomosis • Debridement • Complex seldinger technique Robert Sweet, MD

20. Psychomotor/Visual-Spatial Skills • Advanced skills • Sewing in a “deep hole” • Reconstruction • Adopting techniques to abnormal anatomical/physiologic situations (cognitive too!) • Advanced minimal invasive skills • New adoptions of technology once done with formal training • Occurring in animal labs Robert Sweet, MD

21. With each curricular subtask, choose a learning domain that predominates Robert Sweet, MD

22. Task deconstruction • Methods to do this? • Look at operative reports/atlases • Practical experience • Remember when you were a student • Ask an introspective student/resident to “keep a diary” of • What they are learning • Self-assessment of “level” of comprehension • When they are learning it Robert Sweet, MD

23. Competence vs. Capability COMPETENCE What individuals know or are able to do in terms of knowledge, skills, attitude CAPABILITY Extent to which individuals can adapt to change, generate new knowledge, and continue to improve their performance Brit. Med. J. 323:799-803, 2001 Robert Sweet, MD

24. Learning curves/criterion levels MASTERY PROFICIENCY COMPETENCY Hypothetical Learning Curve for a technical skill set Robert Sweet, MD

25. Simulator Remedial training Learning curves/criterion levels MASTERY PROFICIENCY COMPETENCY Hypothetical Learning Curve for a technical skill set Robert Sweet, MD

26. Learning curves/criterion levels MASTERY PROFICIENCY COMPETENCY Hypothetical Learning Curve for a technical skill set Robert Sweet, MD

27. Metrics for Assessment • Each task has two components • The performance task itself • A clearly defined scoring scheme/rubric Robert Sweet, MD

28. Metrics • Make a list of what is it that is important to measure to determine competence for each subtask? • Establish Criterion levels based on subject matter expert consensus and performance Robert Sweet, MD

29. Metrics examples (adapted from Satava et al. 2002) • Quality metrics • Errors/”near misses”/events/complications • Blunt end vs. sharp end • Missed target • Improper delivery of energy • Bleeding • Final product (leaking or inaccurate) • Patient outcomes • Ability to recover from error • Efficiency metrics • Time to task • Economy of motion • Path lengths • Response latency • Use of resources • Use of assistant(s) • Proper sequence of steps Robert Sweet, MD