Evidence Based Practice in Psychology – Lecture 3

1. Evidence Based Practice in Psychology – Lecture 3 May 31, 2007

2. Basic Concepts in Epidemiology

3. Basic Components of Epidemiological Research • Health Outcome • Explanation • Key Variables: Exposure, disease, control variables • Key Methods: Surveys, interviews, samples, laboratories • Key Designs: Clinical trials, cross-sectional, case-control, cohort

4. Design Considerations • Experimental: RCT’s • Observational: descriptive and analytic • Directionality • Forward: Cohorts and RCT’s • Backward: Case-control • Neither: Cross-sectional • Timing • Prospective: health outcome occurs after study begins • Retrospective: health outcome occurs before study begins

5. More on RCT’s • May be used to test preventative or therapeutic interventions • Key features: • Randomization (control) • Blinding (minimizing bias) • Ethical concerns (stopping rules) • ITT analysis

6. More on Cohort Studies • Best living example: Framingham Heart Study (n = 5100, examined every 2 years) • Information about risk factors and disease states collected • Prospective analysis of health outcomes as a function of risk factors • Cohort studies may also be retrospective • Advantages: • Forward directionality • Exposure, not disease status affects selection, so relatively free of bias • Useful for examining relatively rare exposures • Retrospective study can be inexpensive and quick (e.g., based on employment records or death certificates) • Disadvantages: • Attrition due to migration, lack of participation, withdrawal and death • Inefficient for studying rare disease with long latency • Exposed might be followed more closely than nonexposed, creating spurious exposure-disease relationship

7. More on Case-Control Studies • Subjects selected on the basis of their disease status; first selects cases of a particular disease, then controls without the disease (preferably from same population) • Issue: selection of controls • Advantages: • Good for studying chronic or rare diseases with long latency periods • Require smaller sample sizes than other designs • Disadvantages • Don’t allow several diseases to be evaluated, as do cohort studies • Don’t allow disease risk to be estimated directly because they work backward from disease to exposure • More susceptible to bias

8. Measures of Disease Frequency • Rate • Proportion • Risk (favored for RCT’s and cohort studies) • Odds (favored for case-control, retrospective studies) • Prevalence • Incidence

9. Incidence & Prevalence • Incidence: NEW cases of a disease that develop over a period of time; useful for identifying risk factors and disease etiology; estimated from RCT’s and cohort studies • Prevalence: EXISTING cases of a disease at a particular point in time or over a period of time; estimated from cross-sectional or case-control studies; useful for planning health care services (demand for healthcare) • P = I x D, where D = duration

10. Additional Formulae • P = C/N, where C=existing cases, and N = steady-state population size • CI = I/N, where CI = cumulative incidence (measures risk), I = new (incident) cases; N = size of disease-free cohort • IR = I/PT, where IR = incidence rate, and PT=accumulated person-time information

11. IR = I/PT = 5 new cases/25 disease free person years = .20

12. Rate • Definition: measure of how rapidly health events (e.g., new diagnosis of disease, death) are occurring in a population of interest • Instantaneous: rate at a particular point • Average: rate over time (preferred)

13. Risk • Probability that an individual will develop or die from a given disease, or will develop a health status change over a specified followup period • Assumes that the individual does not have the disease at the start of the followup and does not die from any other cause during the followup • 0< risk < 1 • Necessary to give the followup period over which risk is to be predicted (e.g., 24 months, etc.) • Risk factors • Attribute (e.g., genetic susceptibility, age, sex, etc.) • Exposure (e.g., nutrition, toxicity, injury, etc.) • Risk does not have to refer to disease – could refer to any symptom, side effect, etc. as long as information relevant to such events are measured

15. Risk measures • Attributable risk (AR): Risk in exposed group – risk in nonexposed group • Absolute risk reduction (ARR): similar to AR, but in response to an intervention; it indicates the reduction in risk associated with exposure to an intervention • Etiologic fraction (Population AR): proportion of all cases of a disease that are attributable to an exposure or risk. Proportion of disease in the population that would be eliminated if the risk factor was eliminated or prevented • Relative Risk (RR): Value between 0 and ∞ that indicates the strength of the risk factor and disease outcome. • Calculated by: Risk(exposed)/Risk(unexposed) • Exposure Odds Ratio (OR): estimate of RR derived from a case-control study; similar to relative risk when disease is rare • Number Needed to Treat (NNT): number of individuals that would need to be treated to prevent one adverse outcome in that group of similar individuals at risk of the problem. Establishes benefit of an intervention compared to doing nothing. NNT is the reciprocal of AR or ARR

16. Number Needed to Treat NNT = 1 / |ARR| NNT’s for interventions should be relatively low, for preventative studies a little higher In a randomized controlled trial looking into the long-term outcome for stroke patients treated in stroke units (SU) compared with patients treated in general wards (GW), the mortality rate 5 years after the onset of stroke was 59.1% in the patients treated in SU and 70.9% in those treated in the GW. How many patients need to be treated in stroke units to prevent one additional death? (Stroke 1997; 28:1861-6) NNT = 1 / |.709-.591| = 1 / .118 = 8.5 or 9

17. Number Needed to Harm NNH = 1 / |ARR|, where “risk” is of adverse side effects NNH’s for interventions should be relatively high, at least compared to NNT’s, the higher for more deleterious side effects. In a randomized clinical trial of a drug for movement disorder in Parkinson’s disease, a certain number of adverse effects were recognized. In the treated group 140 of 539 (25.97%) patients developed a clinically measurable memory problem when assessed a year later, while in the nontreated group, 104 of 513 (20.27%) developed such a disorder. NNH = 1 / |.260-.203| = 1 / .118 = 17.5 or 18 This means that 18 patients would have to be exposed to the drug to produce one additional case of memory disorder that would not have appeared naturally in the untreated group

18. Risk v. Rate • Risk required in studies predicting change in health status for individual, or in prognostic studies; rate not useful at the individual level • Risk usually preferred because it is easier to interpret • Sometimes risk is difficult to measure (population studies)

19. Measures of Effect – Risk Ratio Smokers with heart attacks followed over a 5 year period. 5-year death risk: Smokers: 27/75 = 0.36 AR= 0.36 – 0.17 = 0.19 Quitters: 14/81 = 0.17 Estimated RR = .36/.17 = 2.1

20. Measures of Effect – Odds Ratio Case-control study – outbreak of GI disease at resort; cases had diarrhea, controls stayed at resort but did not Proportions: 0.46 0.21 Odds = P/(1-P) Cases = .46/(1-.46) = 0.85 Controls = .21/(1-.21) = 0.27 Odds Ratio = .85/.27 = 3.2 Alternatively: OR = (a x d)/(b x c) OR = (17x26)/(7x20) = 3.2

21. Diagnostic Testing Issues • How well tests perform relative to a “gold standard” is critical to establishing their empirical basis • Often, the issues of cost-effectiveness and risk play a role in test selection • Study design, not just test statistics, is important • Cross validation is key • STARD initiative (next slide) • http://www.consort-statement.org/stardstatement.htm#flow

23. Brunswik Lens Model of Clinical Prediction

24. a b c d Sensitivity = a/(a+c) = 19/36 = .53 Specificity = d/(b+d) = 276/286 = .97 Positive Predictive Power = a/(a+b) = 19/29 = .67 (also known as PTL+) Negative Predictive Power = d/(c+d) = 276/293 = .94 Post-Test Likelihood given Negative Result = 1-NPP = .06 Prevalence = (a+c)/(a+b+c+d) = 36/322 = .11 (pretest probability of d/o) Pre-test Odds = PTP/(1-PTP) = .11/.89 = .12 (.12:1) Likelihood Ratio of Positive Test = [a/(a+c)]/[b/(b+d)] = Sens/(1-Spec) = .53/.03 = 17.67 (17.67:1) Likelihood Ratio of Negative Test = [c/(a+c)]/d/(b+d)] = (1-Sens)/Spec = .47/.97 = .48 (.48:1) Pre-test Odds x LR+ = PPV Pre-Test Odds x LR- = 1-NPV Diagnostic Odds Ratio = LR+/LR- = 17.67/.48 = 36.81

25. SnNout and SpPin • If a test has extremely high Sensitivity and LR+ (say >20), a Negative test result pretty much rules out the target disorder. • If t test has high Specificity and the LR- is very low (say <.05), a Positive test rules in the target disorder

26. ROC Analysis BNP > 76 BNP > 18 Sens = 26/40 = .65; Sens = 35/40 = .88 Spec = 75/86 = .87; Spec = 29/86 = .34

27. Evaluating Studies of Tests • Was an appropriate spectrum of patients included? • (Spectrum Bias) • All patients subjected to a Gold Standard? • (Verification Bias) • Was there an independent, "blind" comparison with a Gold Standard? • Observer Bias; Differential Reference Bias • Methods described so you could repeat test?