Simcoach Standard Patient Studio

1. Simcoach Standard Patient Studio A new kind of Virtual Patient Thomas B. Talbot, MD MedVR GroupInstitute for Creative Technologies

2. Overview • $8M of DoD Advanced Technology R&D Project • A large public trial and distribution of the technology over 4 years • Ambitious scope, yet uses mature technology to mitigate risk • Usable, refined tools for the public • Free • Goal is to create a “critical mass” with the medical education community • Serious emphasis on student assessment • Designed with ‘lessons learned’ from prior work in mind • Desire to replicate the “gold standard” or surpass it

3. What it is: • Conversational Virtual Standardized Patient (The “Clinic”) • Includes plethora of ICT technology: • Avatars, Natural Language Understanding, Artificial Intelligence Dialogue System, Automated Non-Verbal Behavior Generation, Emotional Expression, Trust-based Responses • Works through web-browsers & tablets • Online Authoring System (The Studio) • Guided authoring system targeting medical educators • Leverages prepared ‘personalities’ that are modified • Learner ‘questions’ are mostly pre-packaged because they are the most difficult aspect of patient authoring • Two-phase “open questioning” / “review of system” model • Simple assessment authoring integrated into tools • Shared assets with forum and user rating system

4. The Standard Patient Studio Team • ICT Teams • MedVR Group • Rizzo, Talbot, Williams and others • Screenwriting Team • Simcoach Group • Eric Forbell • Integrated Virtual Human Team • Arno Hartholt • Includes Art Team • Assessment Group • Chad Lane • External Partners • Breakaway Ltd, • Serious game development company, Timonium, MD • USC Keck School of Medicine • Dr. Win May and others • Uniformed Services University School of Medicine • National Capitol Area Simulation Center • Gil Muniz, Alan Liu, et al.

5. What it is not: (lots of things are called “Virtual Patients”) • Online Case Presentation • Those depend mostly on pattern recognition • VR Surgery Simulation • Examples: Immersion Medical, Simbionix • Avatar-based Scenario Simulation • Examples: Pulse!!, HumanSim, others… • Procedurally oriented systems

6. Human Standardized Patients • The “Gold Standard” dialogue-based training system • Rely on student initiative to drive encounter dialogue, ask questions, dynamically follow leads • Close approximation to actual patient encounters • Genuine physical exam and findings • Disadvantages • Expensive, difficult retention • Some pathologies & ages unavailable • Few opportunities to access them • Uncontrolled variation, subjective assessment • There is no desire to replace or eliminate HSPs • HSPs can be retargeted where inherent advantages lie

7. Problems with previous “Virtual Patients”: Lack DialogueNot Readily AuthorableEmphasize things that computer sims aren’t good forLack critical interactive technologiesAvatar is often superfluousLack automated & well designed assessmentLack breadth Are inflexibleLack of content ‘critical mass’ Target populations that do not need the training Too expensive, too proprietary Do not emulate gold-standard approach Attempted things that were too complicated

8. SPS: Achieving A Gold Standard Interaction • Reading the chart • The patient interview (VSP session) • Greeting the patient • Obtaining the chief complaint • Establishing Rapport • Open questioning • Specific questioning (Review of Systems) • Patient Examination (single text page or VSP session) • Physical Examination • Review Laboratory or Radiology information • Make a Diagnosis and Treatment Plan (single menu page) • Counsel the patient on the diagnosis and treatment plan (VSP session) • Assessment (visual summary of skills and progress) • Errors of commission and omission • System’s estimate of progress on learning skills and sub-skills GREY: Lower Emphasis AreasBLACK: Higher Emphasis Areas

9. A Variety of Encounter Types

10. The Patient Interview • Virtual Human Encounter • Full ICT Virtual Human Capabilities • Typed Input, Verbal & Nonverbal Output • Natural Language Recognition • Speech recognition year 3 • On-Screen Educational Guides • Assessment Indicators • Rapport, Achievements • Not used for evaluation oriented patients • Virtual Attending Physician (VAP) • Automatically asks a question when pressed • Conversation • Free-text Question/Answer model • Patient may ask a multiple choice question • Open to Closed Questioning • Answers rapport dependent

11. Virtual Standardized Patient Interview Details • Based on pre-authored “Personalities” • Will create a very well-built out patient with negative/normal responses to a wide variety of medical questions • At least five personality variants will be created • Most educator-authored patients will be based off these personalities • Variety of appearances • Educator will be able to select sex, face, skin, habitus, clothing, etc • Non-verbal tone and baseline rapport is selectable • Respiratory rate/depth can be set or based upon physiology engine data at authoring • Guided authoring by medical educators • First, enter chief complaint, diagnosis • Second, enter patient descriptive dialoge in response to open-ended questioning • Third, select items from “Review of Systems” and history lists, alter responses for appropriate items and select items for evaluation scoring

12. Patient Interview Editor (Open Phase)

14. Virtual Attending Physician (VAP) • A Virtual Human Encounter • White Coat Attending • Simulated Socratic Session • Multiple-choice INOTS-like encounter • Choices with consequences • Unique feedback based upon learner response • Multiple Uses • Patient diagnosis & evaluation • Teaching points • Test for understanding

15. Physical Examination Phase • Different than other physical exams • Features are resource constrained • Will not attempt to do things that can now be done better via other means. • Multimedia interface • Pictures, sounds, videos or animations can be loaded • Think “X-Rays, rashes, auscultation, etc” • Lab/rad results available (physiology engine derived or author entered) • Available via menu or command line • Unique Virtual Human Encounter • Non-verbal, non-dialogue • Responds to commands to perform maneuvers • Neurological & Musculoskeletal focused • “Stand on one leg and close your eyes” • Easy “check the box” based authoring

16. Patient Diagnosis & Ordering • Simple One Page Multimedia Form • Select diagnosis from choices • Select options to order for patient plan • Auto-populates labs/radiology from physical exam phase

17. Patient Counseling • Virtual Human Session • Dialogue Based, Multiple-Choice or both? (undecided) • Resource Constrained • Optional • What the learner should do • Explain the diagnosis & treatment plan • Answer questions • Assess for patient understanding

18. Assessment Screen: Robust & Graphical • Interview Phase • Rapport & Trust • Open/Closed question ratio • Must Ask Items (critical) • Nice to have asked items (secondary) • Acts of commission (bad things you said) • Efficiency • VAP Encounter • Score-based feedback or no Feedback • Physical Examination Phase • Critical & secondary items selected • Efficiency score • Counseling Phase • Undetermined (limited assessment in this version) • Diagnosis & Treatment • Direct feedback on selections made vs. optimal

19. Initial Use Cases: “Bread & Butter” Medical Student / Intern Cases

20. Development Plan • Development Phase (18-24 months) • Construct tools, interface • Build prototype baseline-personality • Initial cases built & secondary personalities authored • Research-based question database optimization • Public Use Phase (24 months) • Open to second-level partners for first three months • Open publically • Public case authoring, student use, forum community • Public presentations & seminars at national events • Reporting

21. Research Questions • Standard Patient Encounters • How useful are virtual standardized patient encounters? • What are the comparative advantages and disadvantages of VSPs compared to human standardized patients? • What kinds of patients, diagnoses or other characteristics are best/worst for VSP encounters? • What phase of the clinical encounter is preferred by educators for a VSP encounter? • How do students respond to the presence of a virtual attending physician (VAP) quizzing and guiding them? Is it useful? • How can VSP encounters best be blended with live standardized patients? • What are the cost implications? • How can use of VSPs free up human resources to expand availability of human standardized patients (HSPs)? If so, what will HSPs being doing more of?

22. Research Questions • Authoring • Can medical educators successfully author compelling VSP interactions when this formerly required teams of experts? • What are the unexpected uses and author populations for this technology? Nurses, allied health, dentists, pharmacists, sales, customer service? • What are the limitations for educator authored VSPs? • What features requires more sophisticated authors? • How can authoring be improved to extend these domains to less sophisticated authors? • Assessment • Are VSP encounter assessments equal or superior to traditional HSP encounter assessments? How? • Is there value to offering a third person perspective replay? • How can an automated assessment be successful in graphically depicting a concise assessment of student performance that has utility for guiding future student performance? • How does VSP encounter performance change with repetition? What is the optimal number of repetitions? • Are VSPs and the assessment good enough for use in state-mandated testing?

23. Research Questions • Capabilities • What capabilities are in greatest practical use? • What capabilities are not being used? Why? • What are the additional capabilities desired by the medical education community? • Other Results • Number of cases and patients authored • Student assessment and performance data • Author and student feedback • Project impact results

24. OPTIONS • Advanced Physical Examination Package ($4M, 2-year effort starting in year 2 or 3) - This option creates a sophisticated physical exam capability that is beyond the scope of the primary effort. Capabilities include: • Large inventory of physical movements triggered by student commands, including sophisticated musculoskeletal and neurological examinations • Advanced graphical ear/nose/throat. Includes advanced exams of the eye, retina, ear canal, eardrum, nares, throat and mouth. • Advanced auscultation system – Use of large scale multifocal recording arrays will truly replicate listening-site specific heart and lung auscultation in real time to include anatomically appropriate and referred sounds. • High fidelity skin pathology system – ability to show a variety of rashes and lesions all over the body. The skin should also alter color based upon anemia. Leverages ICT’s advanced graphics leadership. • Haptic nodule / lymph node palpation – Works through acoustic or physical feedback, depending on device • Physical manipulation and prodding – simulated palpation and reflex hammering • Physical performance maneuvers – simulated orthopaedic tests • Genitalia Examination – surface and external exams only.

25. OPTIONS • Extended Ethnicity, Physical Impairments, and Pediatric Package ($2.5M, 1-year effort starting in year 3 or 4) – This option adds models of different ethnicities, physical impairments, and children of both genders. Capabilities include: • Multiple ethnicity options for all SSPS character archetypes • Multiple physical abnormality options across SSPS character archetypes • Infant Examination – features mother for interview, standard infant examination • Toddler – simulated experience with a toddler in a room • Early and middle childhood school children - dialogue personalities • Early and mid adolescents • Pediatric Physical models- includes age appropriate anatomy, selectable Tanner staging, acne and common rashes • Parent in the room- features two virtual humans in the encounter • Child voices and sounds

26. OPTIONS • Advanced Assessment, Counseling and Decision-making Package ($1.5M, 1-year effort starting in year 3 or 4) – Capabilities include: • Counseling Editor Wizards – create a variety of counseling type interactions • Counseling Assessment System – detailed assessment reporting for counseling • Cost & Resource Assessment – educational feedback on cost and resource efficiency, including educationally-focused cost accounting for student decisions. • Third Person Perspective Replay- with non-verbal behavior depiction of student doctor via avatar • User state sensing – employing Kinect or a webcam sensor to provide assessment data on student arousal, eye contact, facial expression, etc. • Additional Research Year –increases the total SSPS research time and public availability by 50%. • Additional data collection & content creation • Additional content authoring • Prototype for state level VSP testing • Demonstrated integrated VSP/HSP curriculum

27. OPTIONS • Pan-National Cooperative Medical Content Consortium ($2M, 2-year effort starting in year 2) - Capabilities include: • National Virtual Standardized Patient Consortium – a number of medical schools, and perhaps nursing or dental schools from across the country will join forces to create a critical mass of content as a national VSP curriculum. • Twelve New Content Creation Awards - A solicitation will be put out for non-USC schools to participate in $100,000 sub-awards to create standardized patients and contribute to the research effort. Available content is expected to increase 8-10 fold. • National Convention – This will establish a national VSP curriculum that is coordinated across the country. • Expanded Research Report - The report will include more expanded results from the now-larger and broadly based research effort.

28. SUMMARY • The SimcoachStandard Patient Studio uses ICT’s unparalleled technology • The impact of SPS will be revolutionary • SPS will be a special national resource • SPS will closely approximate HSP encounters, but with some advantages • SPS will be useful for both learning & evaluation • SPS will provide objective student data