Admin Part 6: QI, EBM, and all that fun stuff!

1. Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6th, 2007

2. Outline • Evidence based medicine • QI / QA • Stats stuff

3. Levels of evidence Class I: always acceptable/safe/useful Class IIa: acceptable/ safe, results consistently positive Class IIb: acceptable, considered optional or alternative trmts Class III: unacceptable, may be harmful Intermediate: continuing research, no recommendation Strength of evidence A: evidence-based standards (RCT, cohorts, meta-analyses) B: guidelines (case-control, aggregate studies) C: options (x-sectional studies, case series, case reports, panel consensus) Standards of evidence

4. Impediments to decision making • Patient related (content of history, completeness of history, unfamiliar terms, gender/ethnicity-bound, emotion/pain) • Physician related (inadequate knowledge base, lack of data, other department distractions)

5. Tools to assist decision making • Guidelines • Clinical pathways • Computer based protocols • Other evidence

6. Guidelines • “Systematically developed statements to assist practitioners and patient decisions about appropriate health care for specific clinical circumstances” • For diagnostic or therapeutic interventions

7. Guidelines • Pros: Reduce variation in practice, decrease EP burden, promote proven interventions • Cons: may be inadequate, provide contradictory information at times, may have goals other than patient benefit, lack of flexibility, may prevent EP from considering other possibilities, threat of use for legal purposes

8. Development of guidelines • Group to assess nature of evidence • Applicability of evidence to popn of interest • Fiscal implications of recommendations • Assess effect on health system • Technical and administrative support required • Must have fiscal capabilities to develop (or else adopt!)

9. Clinical pathways • “Translation of guidelines into locally developed functional tools that guide the process of care” • Originated in industrial mgmt to reduce variation in production • Aim to promote efficiency and reduce cost • Constant monitoring and data evaluation

10. Efficacy of clinical pathways • Holmboe et al. Use of clinical pathways to improve the care of patients with AMI. Am J Med. 1999. 107:324. • N=1122 pts (10 hosp with pathways, 22 without) • Comparison of hospitals with critical pathways compared to those without • No signif diff in length of hosp stay, mortality, use of proven medical therapies • Did not look at physician compliance with pathways

11. Strategies to change behaviour • Remuneration • Restrict resources • Practice aids (chart reminders, different forms) • Chart audit

12. Computer-based protocols • Require initiative to deviate from recommendation, as opposed to paper-guidelines which require initiative to move to recommended action • May reduce potential sources of errors

13. Improving compliance • Burnstin et al. Benchmarking and QI: the Harvard ED Quality Study. Am J Med. 1999. 107:437. • N=4876 medical records pre-intervention; 6005 post-intervention • Reviewed all medical records for 5 teaching hospitals during 1mo period for 1 of 6 chief complaints and compliance with guidelines; pre and post-intervention • Signif improvement with compliance with guidelines post-intervention • 4% (signif) decrease in pt-reported problems with ED care • The hospital with computer-based guidelines/medical records had greatest improvement in compliance

14. Legal implications of guidelines • Previously thought of as “recommendations” • Implications of deviating from guidelines and potential legal effects are unknown • Guidelines play a “relevant or pivitol role in the proof of negligence” in <7% of lawsuits in US

15. Steps of EBM • Definition of clinical question • Collection of evidence • Analysis of studies for validation, reliability, relevance • Summary of most useful evidence • Application to patient and critical appraisal

16. The Sackett criteria… • http://www.cche.net/usersguides/main.asp • Centre for Health Evidence (on behalf of Evidence-Based Medicine Working Group) • http://www.usersguides.org/

17. Rosens keys to evaluating RCT • When reading a study, look to see that: • pt popn is well defined and comparable to your own pt popn • info on eligible pts that were not enrolled • truly random treatment assignment and adequate blinding procedures • impt confounders were measured and they did not differ signif between groups studied • the % of pts lost to f/u and was a good systematic effort made to perform good f/u • data categorizations that are clear and clinically useful • what agency funded research and what biases might have been introduced

18. Classic hypothesis testing • Null hypothesis—no difference exists between two groups • Alternative hypothesis—groups being compared are different • Calculate p value—based on null hypothesis being true, probability of obtaining observation • Accept/reject null hypothesis—if p<alpha value (usually 0.05) then null hypothesis rejected as false and alternative hypothesis accepted

19. Quality improvement

20. Plan Do Study Act The model consists of two parts: 3 questions & a cycle for learning and improvement • What are we trying to accomplish? • How will we know that a change is an improvement? • What changes can we make that will result in improvement?

21. Plan: • State objectives • Make predictions • Make conditions explicit • Develop plan • Act: • Adopt, adapt or abandon? • Build knowledge sequentially • Study: • Complete analysis • Compare data to prediction • What did you learn? • Do: • Carry out the test • Document problems, observations. • Begin analysis Improvement Cycle“PDSA”

22. Plan Do Act Study “Trial and Learn” Plan - Do - Study - Act measuring results and acting on them Re-evaluate and Continuous Improvement “act, capture the gain and start all over”

23. Rapid and repeated use of the cycle helps teams build knowledge sequentially • Breakthrough • P • P • D • D Results • A • A • S • S • P • P • D • D • A • A • S • S • P • P • D • D • A • A • S • S • P • P • D • D • Learning and improvement • Learning and improvement Theories, • A • A • S • S hunches, & best practices

24. QI Teams • Formed for a specific purpose • Specified life span • Comprised of people who do / know the work • Authority to make changes with approvals

25. What about QA ? • Quality assurance is the steps in place that hopefully help make a process safe • Vs. • Quality improvement which is looking at the system to see what improvements can be made

26. How does QA function? • Safety committee, M+M rounds, near miss/good catch forms… • QI can lead to changes in QA

27. Example • QA process for radiology reports is currently all “discrepant” reports are sent to the duty doc for review (this includes all reports, even if normal, in which the doc did not enter an impression!) • QI project could be developed to look at improving the process, so that only the reports with abnormal discrepant findings are sent… • Any takers?

28. Everyone’s favourite part…STATS!

29. Define each of the following: • Alpha value • Type 1 error • Type 2 error • Power

30. Define the following terms: • Bias • Pre-test probability • Post-test probability • Reliability • Internal/external validity

31. Write equations for each of the following: • Power • Sensitivity • Specificity • Likelihood ratio (and what range is significant?) • Odds ratio • Relative risk • Negative predictive value • Positive predictive value • Number needed to treat

32. Stats tests • Differentiate between: • Students t test and Wilcoxon’s rank sum test • One-way analysis of variance and Kruskal-Wallis test • Chi-squared test and Fisher’s exact test

33. Other important concepts (will make you look smart at journal club)… • Bonferonni correction • Interim analysis of data • Intention-to-treat analysis • A priori probability • Data dredging

34. And now you will WOW Bryan…

35. Questions… ?(please direct all questions to Sarah…)