Evidence Based Practice Module III

1. Evidence Based PracticeModule III Sean Collins, PT, ScD Chair of Physical Therapy University of Massachusetts Lowell

2. Objectives • Demonstrate the ability to • Clinically assess the evidence in terms of validity, clinical importance and applicability • Complete a CAT for the search completed in Session 2

3. Outline • Justified True Belief (1) • Inference (5) • Causation • Hierarchical Structure of Information • Statistical Inference • Threats to Validity • Clinical Significance • Workgroup activity (CAT)

4. Justified True Belief • Evidence based practice is having a justification for the belief you hold regarding the truth of your premises and conclusion Premise 1: If Strength training then increased strength Premise 2: Ms. Jones and strength training Conclusion: Ms. Jones and increased strength

5. Inference (1) • The process of drawing a conclusion • Inductive: Specific observations leading to general conclusions • Deductive: General observations leading to specific conclusions • Abductive Inference: concludes unseen facts, events or causes in the past from clues or facts in the present (IBE) (can be either toward specific or general conclusions)

6. Inference (2) Research – uses inductive & abductive (general) inference Clinical Practice – uses deductive & abductive (specific) inference A proper application of EBP benefits from understanding these underlying inferential approaches

7. Inference (3) All squirrels have bushy tails Fluffy is a squirrel Therefore, Fluffy has a bushy tail All patients with a low MIP improve with IMT Mr. Smith has a low MIP Therefore, Mr. Smith will improve with IMT All patients with disk herniation improve with traction Ms. Jones has disk herniation Therefore, Ms. Jones will improve with traction

8. Inference (4) All patients with a low MIP improve with IMT Do you believe this to be true? Is your belief justified? Mr. Smith has a low MIP Do you believe this to be true? Is your belief justified? Therefore, Mr. Smith will improve with IMT If you believe the premises to be true – then you believe this conclusion is true. If the premises have hesitation – then that hesitation carries to the conclusion.

9. Inference (5) • What causes hesitation regarding the belief being true? • Gaps in understanding of causal mechanisms • Inductive statements (general conclusions) are only probably true – there is a chance that there are situation in which they do not apply • Valid when belief = truth

10. Causation (1) • We are often interested in whether something is the cause of something else • Does X cause Y • Does exposure cause disease • Does intervention cause improvement • Does having X cause Y measurement • Does using X allow measurement of Y • What warrants justified true belief in causal relations?

11. Causation (2) Hill’s Criteria • Temporal Relationship (required) • Strength of Association • Dose Response Relationship • Consistency • Plausibility • Consideration of Alternate Explanations • Specificity • Coherence • Experiment (an approach to establish cause)

12. Hierarchical Structure of Evidence (1) • Systematic Reviews • Randomized controlled trials (experiment) • Longitudinal cohort • Retrospective cohort • Cross sectional • Case – control • Case (cases as in trial and error) • Authority • Tradition

13. Hierarchical Structure of Evidence (2) Evidence of what  truth of a premise Up the hierarchy – increasing the number of Hill’s criteria for establishing cause – that is justified true belief in a causal association

14. Hierarchical Structure of Evidence (3) • Systematic Reviews • RCT • Longitudinal cohort • Retrospective cohort • Cross sectional • Case – control • Case • Temporal Relationship • Strength of Association • Dose Response Relationship • Consistency • Plausibility • Consideration of Alternate Explanations • Specificity • Coherence • Experiment

15. Statistical Inference (1) • Statistical inference – special case of inference • Inductive inference – drawing a general conclusion from specific observations • Statistical inference is used to offer mathematical support of the general conclusion; cannot be certain – but is it probable? Reasonable?

16. Statistical Inference (2) • Related to Hill’s criteria: • Strength of association • Magnitude of effect, change score, effect size • Correlation, regression • T – Tests, ANOVA, Chi Square, etc • Dose response relationship • As above • Alternate explanation (chance) • What you observe – vs. chance (Statistical significance)

17. Statistical Inference (3) Alternate explanation – Statistical significance How likely is it that the X (a finding) we observe occurs by chance? (P value) What is the range of values, based on our observations, that we would expect the true estimate to occur in? (Confidence Interval) How likely is it that you have a disease when a test for the disease is positive? Or that you do not have a disease when the test for the disease is negative? (+LR, -LR)

18. Van Der Tol, 2006

19. Van Der Tol, 2006

20. Smieja et al, 1999 as reported in Fritz & Wainner, 2001

21. Threats to Validity (1) • Valid when belief = truth • Statistical inference  helps rule out chance as an alternate; helps quantify magnitude • Can other factors influence validity? Statistical inference rules out chance, but what if something other than chance leads to untrue observations? If we base our belief on untrue observations then our belief is untrue

22. Threats to Validity (2) • Internal validity – impact the results of the study and its conclusion; confounding factors the interfere with the causal relationship • History • Maturation • Testing • Instrumentation • Regression • Diffusion of imitation of treatments • Compensatory equalization • Compensatory rivalry

23. Threats to Validity (3) • Internal validity – • Operational definitions • Multiple treatment interactions • Experimental bias (Hawthorne effect) • Recruitment bias • Sample bias • External validity • Recruitment & sample bias • Interaction of treatment and selection • Interaction of treatment and setting • Interaction of treatment and history

24. Clinical Significance (1) • If you rule out chance (statistical inference); if you feel confident in the belief (no threats to validity that undermine your confidence) – next question: • What does this mean clinically? Is it clinically significant? • I.e. – I can have a justifiable true belief that Rx X results in an improvement of 6 MWD of 10 feet; but that does not mean an increase of 10 feet in my 6 MWD has any clinical benefit

25. Clinical Significance (2) • Minimally Important Clinical Difference (MICD) (as named) • Number needed to treat (NNT) • Number of patients that need to be treated before a therapist can be sure that one person would improve who would not have improved without that intervention • Number needed to harm (NNH) • As above – but replace improve with harm

26. Clinical Significance (3) • MICD – measurement and value theory driven • NNT & NNH – purely mathematical; however depends on an apriori definition of what it is to “improve” or “harm” which is cannot escape MICD • Good questions – lots to discuss

27. Critically Appraised Topics (CATs) • Title • Author & Date • Clinical Scenario • Clinical Question • Clinical Bottom Line • Search History • Citations • Summary of the study • Design, sample, intervention, outcome measures data analysis • Summary of the evidence • Additional comments