Tim Stratton, Ph.D., BCPS, FAPhA Associate Professor University of Minnesota College of Pharmacy, Duluth

2. 2 Evaluation Team

3. 3 Presentation Overview Problem Statement Needs Assessment Project Goals SISU Network and EMR platform Application of technology to problem Evaluation Plan Lessons Learned

4. 4 Problem Statement How can around-the-clock pharmacist coverage be provided at small hospitals in geographically remote rural communities…

5. 5 Needs Institute of Medicine reports on patient safety related to medication process (in rural hospitals as well) JCAHO pharmacy standards for Community Access Hospitals (CAHs) and National Patient Safety Goals Northeast Minnesota CAHs lack 24/7 pharmacist coverage Relief pharmacists unavailable

6. 6 Project Goals Goal: Provide CAHs with around-the-clock pharmacist coverage Objectives: Utilize telepharmacy technology to connect pharmacy operations of CAHs with a tertiary care facility providing 24/7 pharmacy coverage Hire “circuit rider” pharmacist to provide on-site vacation relief coverage to CAHs

7. 7 SISU Network & EMR Platform “Sisu” A unique Finnish concept – no direct translation Roughly describes special strength and stubborn determination to continue and overcome in the moment of adversity Combination of stamina, courage, and obstinacy held in reserve for hard times SISU Medical Systems: Consortium of 501(c)(3) medical centers in Northern Minnesota (1982) incorporated as 501(e) nonprofit (1997) working together to share information technology resources Uses Meditech to provide technology solutions to its members

8. 8

9. 9 HIT It!

10. 10 Evaluation Approach Pre-post implementation quasi-experimental approach (some post-implementation only) Natural experiment possible Retrospective and prospective data collection Patient admission is unit of measure for clinical outcomes

11. 11 Evaluation Plan Donabedian’s model for evaluating the quality of health care: Structure Process Outcomes

12. 12 Evaluation Plan - History 4/9/2004 Evaluation plan.doc 4/15/2004 Evaluation plan revised.doc 8/3/2004 Evaluation plan response to Feds.doc 10/7/2005 Evaluation plan update.doc

13. 13 Evaluation Plan - Structure RPh coverage achieved for 24/7/365 at subject sites? Facilities have in place medication related P&Ps meeting JCAHO quality standards related to pharmacy services?

14. 14 Evaluation Plan - Process Through bedside use of verification technology, do medication administration practices reach the anticipated level of bedside verification? How does average number of meds per elderly patient differ? Compare med administration error rates (wrong drug, wrong time, wrong patient) and compare to national standards Compare use of preventative services, e.g., immunizations, with national guidelines for patients at risk …

15. 15 Evaluation Plan - Process Time lag between drug orders being written and first dose being administered Number of first doses administered without preliminary review of order by RPh Level of patient satisfaction with the medication process Level of staff satisfaction with the medication process Level of adherence to published drug therapy guidelines.

16. 16 Evaluation Plan - Outcomes Drug overdose or underdose compared to FDA approved dosing ranges Mild/moderate drug related morbidity: drugs D/C’d before 4th dose Severe drug related morbidity: # of ICU admissions/code arrests during hospitalization Anticoagulation management: # of International Normalized Ratio (INR) values out of range …

17. 17 Evaluation Plan - Outcomes Drug-related allergic reactions: # of patients receiving either an H1 antihistamine and/or oral/injectable steroid during hospitalization Appropriate drug use in elderly patients: # of patient falls Noscomial infections: compared to historical data Drug-drug/drug-food/drug disease interactions.

18. 18 Evaluation Plan – Revision I(pre-award decision) Elaboration on data collection approaches to be used for each metric No changes made in intended metrics

19. 19 Evaluation Plan – Revisions II(pre-release of 2nd-year funding) Further elaboration of data collection approaches, responsibilities for each metric List of criteria for project success developed Critical indicator(s) for success identified for each metric Phenomenological approach added to collect “tales from the trenches”

20. 20 Preliminary Results:The Devil’s in the Details… IV labels print remotely, or do they? System flexibility: A double-edged sword Different CAHs enter medication orders under MEDS, FORM(ulary) or PO/IV One CAH chose not to enter “prn” into Meditech med directions dictionary Scheduling Hub hospital scheduled to cover holidays, but what about those Mondays after Sunday holidays?

21. 21 Importance/Feasibility Matrix

22. 22 Evaluation Plan – Revisions III(Test-drive NRC toolkit) “Red zone” metrics Unfeasible to measure Drug-drug/drug-food/drug-disease interactions Drug-related morbidity Number of patient falls

23. 23 Evaluation Plan – Revisions III(Test-drive NRC toolkit) “Red zone” metrics: Project not designed to accomplish Change in average # number of different meds administered to patients aged >= 65 years; Change in % of at risk patients queried about immunization status Change in adherence rate with national treatment guidelines

24. 24 Evaluation Plan – Revisions III(Test-drive NRC toolkit) “Red zone” metrics: Technology not in place in time Beside verification occurs for each patient

25. 25 So What’s Left? Structure: RPh coverage impacted by project? Structure: CAHs have Pharmacy P&Ps in place as intended by licensing/accrediting agencies? Process: What pitfalls are encountered in implementing HIT? Process: Impact on number of 1st doses given without RPh apriori review? Process: Impact on time lag from order written to 1st dose administered? Process: Impact on staff satisfaction with medication process at CAHs Outcomes: Impact of RPh Clinical Interventions

26. 26 Evaluation Plan – Satisfaction Survey Development Pre-post HIT implementation among MDs, Nursing staff, RPhs Web search for similar surveys Existing questions adapted, new questions added Pilot-tested by hospital-based nurses, pharmacists outside of study

27. 27 Evaluation Plan – Chart Review Form Development Retrospective pre-post HIT implementation by Med Records personnel De novo based upon agreed-upon metrics, Evaluator experience in small hospitals Document critiqued by Med Records person, Administrator, RPh of participating CAH Pilot-tested by Med Records, Evaluation Team

28. 28 Evaluation Plan – Lessons Learned Apply KISS principle Follow gut feelings regarding feasibility of collecting data for a given metric Be sure that Project Team members all on “same page” re: project goals Difficult to “let go” of a “good idea”

29. 29