Quasi-experimental study designs in evaluating medicines use interventions

2. Quasi-experimental study designs in evaluating medicines use interventions

3. Lloyd Matowe 1 & Craig Ramsay 2 1 Faculty of Pharmacy, Kuwait University 2 HSRU, University of Aberdeen

4. BACKGROUND • The RCT is seen as the gold standard methodology for evaluating medicine use interventions • There are situations where it is impractical to carry out RCTs • For example in situations where: • It is impractical to randomize subjects • Difficult to find an appropriate control • Intervention has been retrospectively implemented • Quasi-experiments are often used as an alternative

5. Intervention Simple before and after studies Time

6. Intervention Interrupted time series analysis Time

7. Intervention study study control control Controlled before and after design Before After

8. There is suspicion that quasi-experiments are often conducted and analysed inappropriately

9. OBJECTIVE • The objective of this study is to evaluate the use of quasi-experimental designs in medicines use interventions

10. METHODS • A rigorous systematic review • Subset of a large study on the effectiveness of various policy implementation strategies • Search strategies: • EPOC, MEDLINE, EMBASE, HEALTHSTAR, SIGLE • Bibliographies of previously published reviews • Studies included: CBAs & ITS • Interventions: policies/ guidelines to improve medication use • Outcomes- Objective measures of provider behaviour and/ or patient outcome • Time series regression used to reanalyse serial data where reanalysis was necessary

11. RESULTS - 1 • Search strategy identified 150 000 hits • 5 000 hits identified as potentially relevant • 285 studies included • 44 studies were on medicines use • 17 were CCTs or RCTs • 27 studies were quasi-experiments • 15 ITS • 12 CBAs • 24 studies were on multiple interventions • Study areas included: Antibiotics, antipsychotics, analgesics, GIT drugs and anti-hypertensives

12. RESULTS - 2 • Time series • Overall methodological quality poor • 8 of 15 studies analysed inappropriately • Effect size different in all 8 studies • Common errors included: • Failure to test for autocorrelation and trend • Analysis using non time series methods • Failure to ascertain intervention as independent from other interventions

13. Intervention Trends Time

14. RESULTS-3 • Controlled before and after studies • Overall methodological quality poor • 7 of 12 had unit of analysis errors • Other common errors included: • Failure to protect against contamination • Failure to take baseline measurements • Failure to perform power calculation

15. Intervention study study control control Controlled before and after design Before After

16. SUMMARY • Quasi-experiments are often designed analysed inappropriately • Methods and design are often inadequately reported • Study methodological weaknesses are often inadequately reported

17. RECOMMENDATIONS • For ITS studies we recommend: • Use of time series regression • Pre-testing for autocorrelation and trend • Ascertain that the intended intervention was independent of other interventions • For CBA we recommend: • Protection against contamination • Measurement of baseline characteristics • Power calculation • For both ITS and CBAs methodological weaknesses should be reported

18. ACKNOWLEDGEMENTS • I would like to thank Professor Jeremy Grimshaw, from Ottawa Health Research Institute and Center for Best Practices, University of Ottawa, Canada for giving me access to the studies used for this evaluation