The NCSBN National Simulation Study: Results from Year 1

1. The NCSBN National Simulation Study: Results from Year 1 2012 NCSBN Scientific Symposium September 11, 2012 Arlington, VA

3. Study Collaborators Jennifer Hayden, MSN, RN NCSBN, Project Director Pamela Jeffries, PhD, RN, FAAN, ANEF Johns Hopkins University Suzan Kardong-Edgren, PhD, RN, ANEF Boise State University Washington State University Nancy Spector, PhD, RN NCSBN

4. Study Phases

5. Phase II-Multi-site, randomized study of simulation/clinical experiences Purpose Determine if there are differences in clinical competency among graduating nursing students with 50%, 25% or up to 10% of traditional clinical hours substituted with simulation experiences.

6. Study Groups • Control Group (clinical as usual) Up to 10% of clinical time can be in simulation • 25% simulation in place of clinical • 50% simulation in place of clinical

7. Research Questions • Are there differences in clinical competency among graduating nursing students in the three study groups? • Are there differences in knowledge among graduating nursing students in the three study groups? • Are their perceived differences in how well learning needs are met in the clinical and simulation environments among the three study groups? • Are there differences in clinical competency among the three study groups in each of the core clinical courses?

8. Null Hypothesis There are no differences between the study groups in knowledge, clinical competency, and learning needs being met.

9. Substitution Rate • 1 hour of traditional clinical is being replaced with 1 hour of simulation time • Time spent in a simulation lab either as an active participant or active observer of a simulated scenario followed by debriefing • This time in the simulation lab is counted towards required clinical hours for graduation

10. Core Clinical Courses Randomization will be maintained in the core clinical courses • Foundations • Medical/Surgical Nursing • Advanced Medical/Surgical Nursing • Obstetrical Nursing • Pediatric Nursing • Mental Health/Psychiatric Nursing • Community Health/Public Health Nursing

11. Study Will Evaluate: • Clinical competency Creighton Competency Evaluation Instrument (CCEI) • Nursing knowledge ATI Comprehensive Assessment and Review Program • How well student learning needs were met in the clinical and simulation environments Clinical Learning Environment Comparison Survey (CLECS)

12. Defining Simulation Operational definition: An activity or event replicating clinical practice. For the purpose of this study, simulation includes multiple types of scenario based simulation using high fidelity manikins, medium fidelity manikins, standardized patients, role playing, and computer based simulations.

13. NLN/Jeffries Simulation Framework

15. Study Management at each School • Team Leader • Study Team • Trained in the protocol • Trained in the debriefing technique • Large part in the development of the simulation curriculum

19. Clinical Faculty Training • Clinical faculty rate students each week of clinical and each day of simulation using the Creighton Competency Evaluation Instrument • Each semester clinical faculty undergo online training in the use of the instrument • Calculate inter-rater reliability and intra-rater reliability each semester • Clinical faculty review objectives and “norm” what are competent/not competent expectations

20. Simulation Day • Entire clinical group reports to the lab with the instructor • Assigned roles: primary nurse, secondary nurse, family member, documenter, med nurse, observers • 15-20 minutes for scenario followed by 20+ minutes debriefing • Clinical instructors rate their students with the CCEI • 2-3 clinical groups in the lab at the same time

21. Stations • Manikin scenarios • Standardized patients/patient actors • Skills Stations • Group Activities • Layered Learning Critical Thinking Cards • Sensory kits • Group Therapy in a Box • Computerized scenarios • ATI Real Life • Virtual Clinical Excursions

22. Scenarios Available

23. Student “Safety” • Safety data collected each day of simulation and each week of clinical • Oversight Committee at each school Reviews school level safety data, mid-term grades, final grades, ATI scores, feedback from faculty and study team • Data Safety Monitoring Board established Meets monthly to review school level data and national data to ensure participants’ education is not being jeopardized by being in the study

24. Participant Eligibility • 18 yo or older • Provide informed consent • Full-time RN student • Complete the nursing program with study cohort Ineligibility • Licensed as LPN/VN or RN • Accelerated RN student

25. The Students Gender: 86% Female Age: 56% 18-24 28% 25-34 16% 35+ Race: 83% White 8% Black 6% Asian 1% Native American 1% Hawaiian/PI 2% Multiracial Ethnicity: 18% Hispanic

26. Student Demographics Previous Degree: 35% (295) 62% (181) Bachelor’s Degree 41% (120) Associate Degree 3% (9) Master’s Degree Certified Nursing Assistant: 16% Yes Paramedic/EMT: 19 Military Experience: 28 Yes 8 Medical Corps

27. Study Numbers 847 Students consented and randomized 411 ADN 436 BSN 54 Failed a course 39 ADN 15 BSN 96 Withdrew 40 ADN 56 BSN 697 Continuing on to Year 2 332 ADN 365 BSN 82% rate of continuing in the study

28. A closer look at attrition

29. Clinical Instructor Demographics(N=102) Gender: 95% Female Race: 79% White 6% Black 7% Asian 1% Native American Ethnicity: 9% Hispanic Age: 20% Less than 35 24% 35-44 18% 45-54 39% 55+

30. Clinical Instructor Demographics Teaching experience: 94% Yes average 9.0 years (SD 8.7) Current position: 43% Instructor 29% Adjunct faculty 13% Asst or Associate Professor 4% Professor 14% Other

31. Clinical Instructor Simulation Experience 21% No previous experience 55% Observed simulation 47% Run simulation scenarios 50% Debriefed students after simulations 38% Assessed students who have participated in simulations 23% Written simulation scenarios 33% Participated in formal simulation training

32. Clinical Hours Range of total clinical hours w/o capstone: 520-776 hours 25%= 130-194 hours of sim 50%= 260-388 hours of sim

33. What have we learned so far… • Study Model is working • Use of a trained simulation team • Clear objectives • 15-20 minute scenarios • Having clinical instructors observe • Having students observe their peers • Debriefing • Skeptical faculty are now on board

34. Incidental Findings • Informal training/norming of clinical faculty-- more inclusion of adjunct faculty • Bonding of students in their clinical group • Clinical instructors are using DML in their post-conferences • Johns Hopkins University—focus groups of clinical instructors and how this experience is informing their teaching

35. Study Timeline

36. Current Status of Phase II • Study Students are in the senior year of their programs • Students will graduate in May 2013 • We will collect NCLEX pass rates through December 31, 2013 • Final results are expected in Spring 2014

37. Looking Ahead to Phase III • Phase III will follow graduates of the study sites into clinical practice • Purpose of Phase III is to evaluate how well students were prepared for professional practice • Evaluations will include: • Clinical competencies • Safety competencies • Job satisfaction • Retention • Stress

38. What can regulators take away • Who is conducting the sims? Do they have training, what kind? • What kind of debriefing? • How will students spend their time while in sim? • How long are sims? • Are there clear objectives for each sim? • Evaluations/graded? • Is the sim lab accredited?

40. www.ncsbn.org/2094.htmjhayden@ncsbn.org