The ABC’s of ROI (Return on Investment) John Gillespie Laerdal Medical

1. The ABC’s of ROI(Return on Investment)John GillespieLaerdal Medical Welcome to the Indianapolis SUN!

2. Traditional ROI is: “…performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments.” (“Investopedia,” n.d.) Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp

3. Traditional ROI is: “…performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments.” (“Investopedia,” n.d.) “…if an investment does not have a positive ROI, or if there are other opportunities with a higher ROI, then the investment should be not be undertaken.” (“Investopedia,” n.d.) Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp

4. Why is ROI important??? • Sustainability of your simulation program • Help insure that the training dollars are providing value to the organization • Help insure that the learner is receiving value for time spent in training

5. Was/is this a: • Thoughtful investment? • Thoughtless investment? Or

6. Solution Matrix. (2011). Return on investment. Retrieved from http://www.solutionmatrix.com/return-on-investment.html

7. Remember this: “…hospitals typically measure ROI from a business perspective—cost, revenues or operating efficiencies —but many benefits of clinical applications fall into quality and safety realms that do not easily translate into dollars.” (Page, 2010) Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from http://www.hhnmag.com/hhnmag_app/jsp /articledisplay.jsp?dcrpath=HHNMAG/Article/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG

8. Four areas of focus: • Organizational Buy In • Overcoming Barriers to Simulation • Faculty/Personnel Development • Curriculum Integration

9. Organizational Buy In • ROI (Return on Investment) • ROE (Return on Expectation) • Utilization Data Collection • Evaluation Data Collection • Leading indicators are usually pretty close to accurate • Policy and Procedures that are specific to your organization • Care about what keeps your organizational leaders up at night • Create a value statement • Insure that the stakeholder gets a regularly scheduled report

10. But what about ROE?

11. Main principles • “Business/[organizational] objectives are seen as a starting point.” (“Trainingcheck.com,” n.d.) • “’Return on Expectation’ (ROE) is key” (“Trainingcheck.com,”n.d.) • “Collective efforts are needed throughout an [organization] to achieve success.” (“Trainingcheck.com,” n.d.)    Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evaluation. Retrieved from http://trainingcheck.com/training-evaluation /kirkpatrick-ROE-model-of-training-evaluation/

12. Return on Expectation • “Proactive, business partnership approach that unified teams.” • “Defines training as a contributor to key business results.” • “Value defined by business stakeholders in cooperation with training.” • “Focus on comprehensive evidence and a compelling story of value.” • “Easy to understand, flexible, and cost-effective.” Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expectations is getting so much attention. Training + Development. Retrieved from http://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm

13. Data Collection • To prove either ROI or ROE, you must have data to base your return on.

14. Types of data collection to support ROI & ROE • ROI in Healthcare- Using data to demonstrate a positive impact on the organization through the use of simulation • This can include unexpected results • ROI in operations- Use, or utilization • Tool for determining when you are under utilizing/over utilizing your resources • ROI in customer service- Were our customers satisfied with the simulation experience? • Can become a tool for measuring the effectiveness of your instructors/simulations • ROI in academic learning- Assessment • Process driven checklist type of scoring

15. Measuring Outcomes • The effectiveness of your program is only as relevant as the effectiveness of the tool(s) that you are using to measure it by. • Using the Kirkpatrick model, as one example, you can and should measure the effectiveness of the learning objectives met right after the simulation and then again 6 months later. • Is the behavior reflecting what was learned during simulation (Debriefing)? Report this back to the stakeholder. Are you indicating a beneficial return on investment?

16. Monthly Log WakeMed-Amar Patel

17. Monthly Log WakeMed-Amar Patel

18. Monthly Log WakeMed-Amar Patel

19. Monthly Log WakeMed-Amar Patel

20. Month End Data WakeMed-Amar Patel

21. First Quarter - Raw • WakeMed-Amar Patel

22. First Quarter - Graphed WakeMed-Amar Patel

23. Targets WakeMed-Amar Patel

24. How does this data help? WakeMed-Amar Patel

25. An actual data collection tool……

26. Back to the ROI…..

27. How is simulation use valuable? • Value 1- Simulation is only valuable if you use it. • A simulator that is unused/under utilized has no value. • Worse, it is a cost center and a financial drain on the organization.

28. How is simulation use valuable? • Value 2.- Educationally effective simulation not only allows the learners to “connect the dots”, but it also can change behaviors in an organization. • The result can allow you to focus your training dollars to effect • Reduce your malpractice costs • Reduce time spent in training • Improve patient outcomes

29. How is simulation use valuable? • Value 3.- Use as a recruiting tool to draw in candidates that increase the overall value of your organization.

30. How is simulation use valuable? • Value 4.- Repeatability regardless of the patient census at any given moment. • Allows for a consistent product to be delivered to the learners.

31. References Best Practice Committee. (2002). The Value of IT investments: It’s not just return on investments. Retrieved from http://www.cio.gov/documents /thevalueof_it_investments.pdf Big Dog & Little Dog’s Performance Juxtaposition. (n.d.). Kirkpatrick’s four-level training evaluation model. Retrieved from http://www.nwlink.com/~donclark/hrd /isd/kirkpatrick.html Cooke, M., Irby, D., Sullivan, W., & Ludmerer, K. (2006). American medical education 100 years after the Flexner report. The New England Journal of Medicine, 355(13), 1339-1344. Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expectations is getting so much attention. Training + Development. Retrieved from http://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm Markow, M. (2011). Business terms glossary. BusinessWings. Retrieved from http://www.businesswings.co.uk/articles/Business-terms-glossary Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from http://www.hhnmag.com/hhnmag_app/jsp /articledisplay.jsp?dcrpath=HHNMAG/Article/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG Robert Wood Johnson Foundation. (2011). Glossary of healthcare terms. Quality/Equality. Retrieved from http://www.rwjf.org/qualityequality/glossary.jsp SBA.gov. (2011). Essential elements of a good business plan. Retrieved from http://www.sba.gov/category/navigation-structure/starting-managing-business /starting-business/writing-business-plan/essential-elements-good-busines Smith, R. (2009). Embracing game technology for medical education. Retrieved from http://www.peostri.army.mil/CTO/FILES 2009_RSmith_MT3.pdf Solution Matrix. (2011). Return on investment. Retrieved from http://www.solutionmatrix.com/return-on-investment.html Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evaluation. Retrieved from http://trainingcheck.com/training-evaluation /kirkpatrick-ROE-model-of-training-evaluation/ Research in Simulation: Using Data to Help Improve a Return on InvestmentAmar P. Patel, MS, NREMT-P, CFC Laerdal SUN Conference Foxwoods 2012