A Cluster Controlled Trial Comparing Three Methods of Implementing Practice Guidelines for Children with Croup

1. ‘A cluster controlled trial comparing three methods of implementing practice guidelines for children with croup’ Knowledge Translation Canada Research Network May 14th, 2009

2. Pediatric Emergency Research Canada

3. Theoretical Model • The ‘Figure Eight’ Iterative Loop of Research • (modified from Tugwell et al 1985) #1 BURDEN OF ILLNESS #5 SYNTHESIS & IMPLEMENTATION #2 AETIOLOGY OR CAUSATION CR KT #6 MONITORING OF PROGRAMME #3 COMMUNITY EFFECTIVENESS #4 EFFICIENCY #7 REASSESSMENT

5. Clear EvidenceEffectiveness of corticosteroids • Five fold reduction in intubations • 33% reduction in hospital length of stay • 50% reduction in hospital admissions • 50% reduction in return for care

6. KT Initiative • Started in 2000 • Joint Collaboration with Alberta Medical Association (AMA) • Test several methods for implementing CPG • Funded by AMA & CIHR • Seven Year Long Project

7. Study Design ‘The overall objective of this study is to identify, from a societal perspective, the costs and associated benefits of three strategies for disseminating and implementing a practice guideline that addresses the management of croup.’

8. Development of Guideline • Multi-disciplinary, wide-geographic representation • AGREE Guidelines • Treatment • Steroids for everyone! • Epinephrine for severe • Other treatment is unproven

9. Development of Guideline • Indications for disposition • Discharge • Admission • Persistent stridor & chest-wall indrawing at rest • After Rx with steroids, observe 4 hrs before admitting • Psychosocial • Notification/Transfer PICU

10. Implementation • Three Interventions • First “Standard” • Guidelines posted on Web and sent in mail to all Physicians

11. Implementation (cont.) • Second “Saturn” • Focused on all health care professionals (‘local champion’) • Interactive teaching session • Standard treatment protocol (reminder system) • Outreach visits (used minimally)

12. Implementation (cont.) • Third “Cadillac” • Focused on physicians • Opinion Leader -- Endorsed mail-out, interactive teaching session, formal & informal peer interactions Lomas et. al. JAMA. 1991 May 1;265(17):2202-7

13. Evaluation part 1 • Utilization rates • From Administrative Data (Apr 94-Mar 06) • Hospitalization • ED visits • Physician billings • Croup cases identified using ICD9/10 codes, extracted, data sets linked

14. Evaluationpart 1 • Utilization rates (cont.) • Create ‘disease episodes’ (first & subsequent contact up to 14 days) • Assign physicians to hospitals based on practice patterns • Assign disease episodes to hospitals based on MD who assesses first encounter

15. Evaluationpart 1 • Utilization rates (cont.) • Primary Outcome – hospital days per 1,000 disease episodes • Other outcomes - hospital admissions and physician visits per 1,000 disease episodes (DE) • Other outcomes – adverse events (using ICD9/10 codes)

16. Selection of Participating Hospitals • Utilization rates (cont.) • Rank ordered 107 Alberta hospitals • Hospital admissions per DE • Average annual number of DE • Approached hospital administrators to participate in study until 24 accepted

17. Intervention Arms • 24 total hospitals • Range from urban general to small rural hospitals • Matched-pair randomization • # croup episodes • steroid use • % admission • 8 per intervention arm

18. Evaluationpart 2 • Medical Chart Audit • 3 trained research assistants • Direct entry into standardized Access data base • Established inter-rater reliability • Review all hospital admissions • Review all ED visits in small hospitals & random sample in large hospitals

19. Evaluationpart 2 • Medical Chart Audit (cont.) • April 1994 – March 2006 • Extracted data • Patient demographics • Clinical parameters to estimate severity • Laboratories, radiography • Medical treatment & timing

20. Evaluationpart 2 • Medical Chart Audit (cont.) • Secondary Outcomes • % treated with steroid in ED or in hospital • % treated with steroid at least 3 hours before admit decision • Time to treatment with steroids in ED and in hospital

21. Evaluationpart 2 • Medical Chart Audit (cont.) • Other Outcomes • Review potential cases of adverse events identified by ICD coding • Audit medical examiner cases (age, broad cause – respiratory)

22. Evaluationpart 3 • Prospective follow-up of families of children with croup in 24 hospitals • Nov 2001 – March 2006 • Daily telephone calls until symptoms resolve • Document duration of symptoms, family burden (e.g. stress, sleep-lost), disease-related financial expenditures, work missed

23. Evaluationpart 3 • Prospective follow-up (cont.) • Other outcomes • Croup symptoms on day 1, 2, & 3 • Maternal stress on day 1, 2, & 3 • Total sleep lost for disease episode

24. Time Frame • Implementation • Autumn 2003 • Intervened in 21 out of 24 hospitals • Autumn 2004 • Intervened in other 3 hospitals

25. Time Frame • Analysis • Pre – 6 years prior to implementation at each hospital • Implementation year • Post – 2 years after implementation for 21 hospitals • 1 year for three hospitals

26. Statistical Analysis • Primary Outcome Linear mixed model (Laird-Ware approach) • Random effects • Hospital • Year within hospital • Admitting MD • Fixed effects • Intervention • Year • Secondary Outcomes – similar

27. Economic Analysis • Societal Perspective • Provider costs from providing health care (administrative data) • Provider costs from development & implementation of pathways • Costs to family stemming from child’s disease (prospective follow-up)

28. Qualitative Analysis • Post-implementation • Small group interviews of staff (almost all nurses) • 12 of 24 hospitals purposefully selected • Standard vs. Saturn vs. Cadillac • Good vs. Poor Responders

29. Results Initial Baseline Data

30. Alberta-wide Utilization Data (1994-2000)

31. % Steroid % EPI % bAgonist % AB % DC Small OPT 20% 14% 20% 28% 7% IPT 29% 47% Mid OPT 21% 16% 27% 25% 3% IPT 57% 52% Large OPT 47% 14% 24% 10% 1% IPT 83% 57% Treatment in 24 Study Hospitals (1994-2000)

32. Primary Outcome

33. Adjusted Mean Hospital Days per 1,000 Disease Episodes

34. Adjusted Mean Differences between Intervention Arms

35. Hosp Days per 1,000 DE Adjusted Mean Differences per Yr

36. Secondary Outcome % treated with steroids

39. Adjusted Odds Ratio for Receiving Steroids

40. Odds Ratios of Intervention Arms for Receiving Steroids

41. Economic Analysis Provider Health Care Costs

42. Adjusted Mean Hosp Costs per 1,000 Disease Episodes

43. Hosp Costs per 1,000 DE Adjusted Mean Differences per Yr

44. Summary Intervention Costs Costs in Dollars

45. Summary of Results • Still partial; includes only: • Primary Outcome • One of three Secondary Outcomes • Only Provider & Intervention Economic Costs

46. Summary Results (cont.) • Primary Outcome - Hospital Days per 1,000 DE • Gradually lowered over 9 years • Post Yr 2 is approximately ½ that of Pre Yr 6 • There is no difference between intervention arms

47. Summary Results (cont.) • Secondary Outcome - % of ED & Hospitalized Patients treated with Steroids • Gradual increase in % across 9 yrs. • Increase is greater in smaller hospitals

48. Summary Results (cont.) • Secondary Outcome (% steroid) • Cadillac steroid use in intervention year was significantly different from Standard steroid use • Trend for this pairing in Post Yrs 1 & 2 • Saturn as compared with Standard steroid use did not reach statistical significance at any time

49. Summary Results (cont.) • Provider Costs • Gradually lowered over 9 years • There is no difference between intervention arms

50. Conclusions • Strong secular trends • Decreasing # of hospital days • Increasing use of steroids • Decreasing health care costs • Active Implementation • Minor increase in steroid use • No effect on hospital utilization