Introduction

1. Routine use ofthe “ADE Scorecards”,an application forautomated ADE detection,in a general hospital Emmanuel Chazarda, Michel Luyckxb, Jean-BaptisteBeuscartc, Laurie Ferreta, RégisBeuscarta a Public Health Department, Lille University Hospital; UDSL EA 2694; Univ Lille Nord de France; F-59000 Lille, France. b Hospital Pharmacy, Lille University Hospital; UDSL EA 4481; Univ Lille Nord de France; F-59000 Lille, France. c Geriatrics Department, Lille University Hospital; UDSL EA 2694; Univ Lille Nord de France; F-59000 Lille, France.

2. Introduction Adverse Drug Events The PSIP European Project The “ADE Scorecards” Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

3. Adverse drug events • ADEs = Adverse Drug Events • Several definitions. Institute of Medicine (2007): • “An injury resulting from the use of a drug” • “An injury due to medication management rather than the underlying condition of the patient” • Epidemiological data: • 98,000 deaths per year in the US • An ADE would occur in 5-9% of inpatient stays • Two fields of research: • Retrospective ADE detection • Prospective prevention of ADEs by CDSS (clinical decision support systems) Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

4. The PSIP European ProjectPatient Safety Through Intelligent Procedures in Medication • Hospitals: • Lille University Hospital (F) • Rouen University Hospital (F) • Denain general hospital (F) • Hospitals from the Region Hovedstaden (Dk) • Industrial partners: • Oracle (Europe) • IBM Denmark (Dk) • Medasys (F) • Vidal SA (F) • KITE solutions (I) • Ideea Advertising (Ro) • Academic partners: • Aristotle Thessaloniki University (Gr) • Aalborg University (Dk) • UMIT – Innsbruck University (Au) • Funded by the European Research Council, 7th framework program(agreement N°216130) Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

5. The “ADE Scorecards”, a tool for automated ADE detection in EHR EHR ADE de-tection rules 236 validated complex rules Database Computation step Real past inpatient stays Statistics and automated filters are site-dependant List of potential ADE cases Contextualized statistics Web-based display tool Routinely-collected data (diags, lab, drugs, etc.)./!\ADEs are not flagged. Multilingual (En, Fr, Dk, Bu) • Statistics on ADEs • Description of rules • Cases review Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

6. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

7. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

8. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

9. Number of cases per month Histogram of appearance delay Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

10. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

11. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

12. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

13. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

14. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

15. 83 Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

16. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

17. The ADE Scorecards, a tool for automated ADE detection in EHR • Usage: • Installed in 5 hospitals (2 Danish, 2 French and 1 Bulgarian) • Routinely used by the physicians and pharmacists of a French general hospital during three years • Objective: show the results of its use in real-life situation Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

18. Material and methods Quantitative analysis Qualitative analysis Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

19. Quantitative & qualitative analysis • Analyzed data: January 2007 to August 2012 • Description: Patients, medical background (95% confidence intervals) • Comparisons: • Between medical departments (displayed here: Gynecology-Obstetrics vs. Cardiology) • Chi-2 tests & Student’s t-tests (with =5%) • Focus on 2 outcomes: • Hyperkalemia: defined as K+>5.5mmol/lmay induce lethal cardiac rhythm troubles • INR increase: defined as INR≥5 (INR=international normalized ratio) due to VKA overdose or interaction (VKA=vitamin K antagonist)may induce a severe hemorrhage • Observation of the daily use by a human-factors specialist and a pharmacist Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

20. Results Characteristics of the patients Potential ADEs with INR increase Potential ADEs with hyperkalemia Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

21. Characteristics of the patients(n=73,836 inpatient stays) Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

22. Potential ADE cases with INR increase (INR≥5) Estimated proportion:0.99% [0.89%;1.09%] Interesting data: Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

23. Potential ADE cases with hyperkalemia (K+>5.5mmol/l) Estimated proportion:2.03% [1.89%;2.18%] Interesting data: Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

24. Results of qualitative evaluation and Discussion ADE Scorecards Good accuracy (precision 50%, recall 95%) User-centered design, good evaluation (questionnaire) Regular spontaneous use of the soft Prescription analyses Pharmacist Weekly meetings Improvement of the drug management Physicians Patient records screening Physicians in charge of quality of care Morbidity and mortality reviews Decrease of morbidity Initial design: Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

25. Results of qualitative evaluation and Discussion • Faster preparation • Real local cases (not perceived as theoretical) Faster preparation for drug-related morbidity Prescription analyses Pharmacist Weekly meetings Improvement of the drug management? Physicians Patient records screening Physicians in charge of quality of care Morbidity and mortality reviews Decrease of morbidity? support to a global quality improvement approach: Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

26. Thankyou for your attention! • Free demonstration: http://psip.univ-lille2.fr/prototypes/public/ • The research leading to these results has received funding from the European Community’s Seventh Framework Program (FP7/2007-2013) under Grant Agreement n°216130 - the PSIP project. • Contact: emmanuel.chazard@univ-lille2.fr Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

27. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

28. The “ADE Scorecards”General procedure Databases of inpatient stays ADE detection rules Identification of potential ADE cases Statistics computation Data Mining &expert validation Tool for comprehensive visualization: the ADE Scorecards Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

29. Available data: ~155,000 inpatient stays from 6 hospitals (F, Dk, Bu) Diagnoses E119Diabetes I251Athérosclérosis I10Arterial hypertension N300Cystitis Data aggregation, Mappings,data management Demo. & Admin Age 80 Man?0 Dead?0 Length of Stay9 (…) Simplified database with potential outcomes and context Lab results NPU03230Potassium Data mining(decision trees, etc.) Procedures ZZBQ002 Thorax radiography Administered drugs ADEcases ADE rules Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

30. Which methods forretrospective ADE detection ? • Reporting systems: • Based on spontaneous case reports • Mandatory, but underreporting bias: less than 5% cases are declared! • Expert-operated chart reviews • Reference method, expert validation • Time consuming: 30 min per case, and some ADEs are very rare… • Objective: using data reuse & data mining to: • Automatically identify past ADE cases • Generate ADE detection rules • Computing probabilities of occurrence Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

31. Drugs Administrative data 88 years old woman Acetaminophene VKA Vitamin K Statin Red blood cells Diagnoses I10 Arterial hypertension Z8671 Personal history of myocardial ischemia I620 Non-traumatic subdural hemorrhage Laboratory results Medical procedures ABJA002 Drainage of an acute subdural hemorrhage, by craniotomy FELF001 Transfusion INR Free-text reports Hemo-globin Discharge letter Surgical report Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

32. Material and Methods Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

33. Results Example of decisiontree & interpretation The ADE Scorecards, a tool for automated ADE retrospectivedetection Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

34. ResultsExample of decision tree (1) • VKA= vitamin K antagonists (anticoagulant) • INR= international normalized ratio. Evaluates VKA activity • INR>5 => risk of hemorrhage • The tree attempts to explain INR>5 VKA no yes f=0.1 Butyrophenone discontinuation no yes f=0 f=0.25 Hypoalbuminemia no yes f=0.4 f=0.2 f=0.05 f=0.5 Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

35. ResultsExample of decision tree (2) • VKA &butyrophenone discontinuation P=0.4 • VKA & no butyrophenone discontinuation &hypoalbuminemiaP=0.5 • VKA & no butyrophenone discontinuation & no hypoalbuminemiaP=0.05 • no VKA  P=0 We obtain 4 rules. 2 of them are associated with increased risk of hemorrhage => 2 ADE detection rules 4 1 3 2 Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

36. VKA VKA VKA VKA Buty. Buty. INR INR INR INR Step 1: normal VKA intake Step 2: butyrophenone=> transit acceleration=> too low INR Step 3: increased intake of VKA=> normal INR Step 4: buty. discontinuation=> normal transit=> too high INR ResultsExample of decision tree (3) Butyrophenone:neuroleptic drugs, may accelerate the intestinal transit Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

37. Normal state:99% of the VKA bind to albumin.Only 1% of VKA are biologically active. The intake is based on it. Hypoalbuminemia:decrease of the bound fraction,increase of the non-bound fraction=> too high INR (with constant intake) ResultsExample of decision tree (4) Albumine = plasmatic protein to which VKA bind. Only the non-bound part is biologically active. Serum albumin VKA Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

38. 52.0% 95.1% ResultsEvaluation of the ADE retrospective detection • Complete 2010 year of one hospital • Number of stays : 14,747 • Number of hyperkalemia cases : 117 (7.93‰)  exhaustive review • Result • Precision 39/75= 52.0% • Recall 39/41= 95.1% • Harmonic mean 67.2% • Number of reported cases 0/41= 0% Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

39. Prescription Prescription Prescription Hospital information system, databases Retrospective detection of ADEs • Retrospective identification of past ADEs, although no explicit signal exists in the data Inpatient stay Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

40. Change of the drug prescription Alert method Prospective prevention of ADEs • Alert generation, before the ADE occurs, in order to prevent it. Prescription Prescription Inpatient stay Hospital information system Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

41. Statistiques sur les EIMdans la base de données PSIP *: le nombre d’événements est rapporté au nombre total de séjours, alors que les nombres suivants sont rapportés aux séjours de plus de 2 jours uniquement **: ces nombres sont extrapolés depuis un échantillon Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

42. Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

43. X all departments X surgery X gyneco-obstetrics X all dpts X medicine A X medicine B X pneumology Y all departments Y apoplexy Y cardio & endocrinology Y geriatrics Y gynecology Y intensive care unit Y internal medicine Y obstetrics Y orthopedics Y rheumatology Y urology Z all departments W all departments Pour chaquerègle, les statistiquescontextualiséessontcalculéesdanschaque service Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

44. Prévention prospective des EIM Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

45. Prévention prospective des EIMApproche de PSIP • Ex : AVK & IPP  risque hémorragique • Implémentation classique : • Implémentation PSIP : Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

46. Prévention prospective des EIMApproche de PSIP • Développement de plusieurs systèmes d’aide à la décision : • Prototype IBM • Prototype Medasys • Simulation web d’ordonnance • Caractéristiques majeures : • Alertes filtrées statistiquement, contextualisées moins d’alertes, plus pertinentes • Règles raffinées (segmentation statistique) prédiction du risque plus pertinente • Méthodes d’alerte moins interruptives, plus acceptables Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

47. Data reuse in insurance companies… Daily feedingand updating Demographic data Transactional activities.The company: Contributions data (incomes) Recruits and follows customers Banks insurance premiums Accidents database (outcomes) Pays out claims Data transformation Model for predicting individual risk Statistical analysis Nearly-custom database How much should MrSmith pay for his car insurance? Decision Personalized insurance premiums Routine activities: Reuse of the data: Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

48. Definition of “big data” 2-Many variables 4-Many tables & relationships 1-Many records 5-Variables with repeated measurements 3-Many possible values for qualitative variables “big data” is generally a property of the routinely collected data that can be reused “Big” can be understood through 5 dimensions: Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

49. Challenges in data reuse Here? Not really… Data mining techniques( statistical methods) are used, but not specific. Here? Yes, mainly! The decisions that are taken for the data transformation process have a critical effect. Transactional database 1 Transactional database 2 Transactional database 3 Data transformation Scientific question Nearly-custom database Results Statistical analysis Interpretation Here? Partially… Significant tests are nearly always observed in Big Data: correct the  risk, consider the effect size. Cf. post. Knowledge Where is the secret of a successful data reuse? Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

50. Data reuse of EHR(Electronic Health Records) Diagnoses E119Diabetes I251Athérosclérosis I10Arterial hypertension N300Cystitis Procedures ZZBQ002 Thorax radiography Demo. & Admin Age 80 Man?0 Dead?0 Length of Stay9 (…) Lab results NPU03230Potassium Administered drugs Free-text reports Medical devices?? Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.