1 / 65

Statistics for clinicians

Statistics for clinicians.

buzz
Download Presentation

Statistics for clinicians

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistics for clinicians Biostatistics course by Kevin E. Kip, Ph.D., FAHAProfessor and Executive Director, Research CenterUniversity of South Florida, College of NursingProfessor, College of Public HealthDepartment of Epidemiology and BiostatisticsAssociate Member, Byrd Alzheimer’s InstituteMorsani College of MedicineTampa, FL, USA

  2. SECTION 7.1 Introduction to Meta-Analysis Meta analysis and measures of public health impact

  3. Learning Outcome: Recognize the methodological basis for conducting meta-analyses.

  4. Key Questions Question 1: Why is it difficult to appropriately evaluate/synthesize findings from multiple sources of research? Question 2: Why do we often arrive at different or incorrect conclusions in groups of studies, and among individual studies?

  5. Traditional Definitions Meta-Analysis:A quantitative approach for systematically combining the RESULTS of previous research in order to arrive at conclusions about the body of research. (e.g. Determination of risk ratio and confidence interval across studies – Is Intervention A better than Intervention B?)

  6. Traditional Definitions Pooled-Analysis:Pooling of PRIMARY DATA from multiple studies for the purpose of conducting an analysis of the enlarged data set. (Often referred to as “meta-analysis using individual patient data”).

  7. The Need for Meta-Analysis 1940:2,300 Biomedical journals 2011: 39,529 journals listed in PubMed 2011: More than 10,000 randomized clinical trials conducted each year Similar proliferation of journal articles for social science disciplines

  8. Range of Reaction to Meta-Analysis Supportive: Cook, Guyatt (1994) The professional meta-analyst: an evolutionary advantage Rosendaal (1994) The emergence of a new species: the professional meta-analyst Feinstein (1995) Meta-analysis: statistical alchemy for the 21st century

  9. Range of Reaction to Meta-Analysis Neutral: Meinart (1989) Meta-analysis: science or religion? Spector, Thompson (1991) The potential and limitations of meta-analysis. Bailar (1997) The promise and problems of meta-analysis.

  10. Range of Reaction to Meta-Analysis Critical: Eysenck (1978) An exercise in mega-silliness. Chalmers (1991) Problems induced by meta-analysis Thompson, Pocock (1991) Can meta-analysis be trusted? Greenland (1994) Can meta-analysis be salvaged? Shapiro (1994) Meta-analysis/schmeta-analysis

  11. SECTION 7.2 Steps Involved In Meta Analysis

  12. Learning Outcome: Demonstrate the primary steps involved in conducting a meta-analysis

  13. Steps in a Meta-Analysis Step 1: Identify studies with relevant data Step 2: Define inclusion and exclusion criteria for component studies Step 3: Abstract the data Step 4:Analyze the data

  14. Steps in a Meta-Analysis Step 1: Identify studies with relevant data • Completeness of information (e.g. published and unpublished reports) • Specificity of hypothesis (e.g. similarity of treatments and/or exposure) • Choice among multiple publications

  15. Steps in a Meta-Analysis Step 2: Define inclusion and exclusion criteria for component studies • Study designs (experimental, observational, etc.) • Years of publication of study conduct • Languages (e.g. English language only) • Incomplete data (e.g. loss to follow-up) • Quality (e.g. subject selection, blinding, treatment compliance, statistical methods, etc.)

  16. Step 2: Define inclusion and exclusion criteria for studies

  17. Steps in a Meta-Analysis Step 3: Abstract the data • Select the desired measure of effect and reported estimate (e.g. odds ratio, standardized mean difference, correlation coefficient) --- unadjusted --- adjusted for age only --- adjusted for multiple confounders, etc. • Are the data available for subgroup analyses?

  18. Selection of Component Studies Concern: Publication Bias Statistically significant results 3x more likely to be published than papers affirming a null result.(1) Most common reason for non-publication is investigator declining to submit results(2) e.g.: --- loss of interest in topic --- expectation that others will not be interested in null results Also known as “file drawer” bias. 1. Dickersin K, Chan S, Chalmers TC, et al. Publication bias and clinical trials. Controlled Clin Trials 1987; 8: 343-53 2. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337:867­72.

  19. http://clinicaltrials.gov/ct2/home ClinicalTrials.gov is a registry and results database of federally and privately supported clinical trials conducted in the United States and around the world. ClinicalTrials.gov gives you information about a trial's purpose, who may participate, locations, and phone numbers for more details. This information should be used in conjunction with advice from health care professionals. Who is responsible for registering the trial? In most cases, the Sponsor of the trial as defined by FDA regulations [21 CFR 50.3(e)] has the obligation to register the clinical trial with ClinicalTrials.gov.

  20. Steps in a Meta-Analysis Step 4: Analyze the data (statistical methods) • Fixed effects • Random effects • Bayesian approach (not discussed)

  21. SECTION 7.3 Statistical Methods in Meta-Analysis

  22. Learning Outcomes: Recognize analytical considerations in conducting a meta-analysis Calculate and interpret a summary odds ratio using the Mantel- Haenszel method

  23. Step 4: Analyze the data (statistical methods)

  24. Fixed Effect Method: Meta-Analysis --- The “within-study”variance is used as the weighting factor for each study Example: MH method to estimate the summary odds ratio: sum(weighti x ORi) ORMH = ---------------------------- sum(weighti) where weighti = 1 / variance Note – variance is for log(OR)

  25. Example: Mantel Haenszel Summary Odds Ratio Example Calculations for Study #1 OR = (Odds of exposure among cases / Odds of exposure among controls) OR = (49 / 67) / (566 / 557) = 0.72 Variance - (log)OR = ((1/N1) + (1/N2) + (1/N3) + (1/N4)) Variance – log(OR) = ((1/49) + (1/67) + (1/566) + (1/557) = 0.0389 Weight = 1 / variance = 1 / 0.0389 = 25.71 Weight x OR = 25.71 x 0.72 = 18.50 sum(weighti x ORi) 821.24 ORMH = ---------------------------- OR = ---------------------- = 0.90 sum(weighti) 910.56

  26. Practice: Calculate the Mantel-Haenszel Summary Odds Ratio for the Studies Listed Below OR = (Odds of exposure among cases / Odds of exposure among controls) Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4)) Weight (W) = 1 / variance sum(weighti x ORi) ORMH = ---------------------------- OR = ---------------------- = sum(weighti)

  27. Practice: Calculate the Mantel Haenszel Summary Odds Ratio for the Studies Listed Below OR = (Odds of exposure among cases / Odds of exposure among controls) Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4)) Weight (W) = 1 / variance sum(weighti x ORi) 122.75 ORMH = ---------------------------- OR = ---------------------- = 1.32 sum(weighti) 93.06 Across the 6 studies, the summary odds ratio is 1.32

  28. Fixed Effect Method: Meta-Analysis Since larger studies are more precise the smaller studies (hence smaller confidence intervals), the fixed effect approach gives more weight in the analysis to the larger studies. The inference applies to only the studies included in the meta-analysis.

  29. Random Effects: Meta-Analysis --- The “within-study”and “between-study” variances are used as the weighting factor for each study. Example: Dersimonian and Laird method: sum(w*i x ln ORi) ln ORDL = ---------------------------- sum w*i where w*i = 1 / [D + (1 / wi)]

  30. Random Effects: Meta-Analysis For the term in w*i wi = 1 / variance (within-study variance) [Q – (S – 1)] x sum wi (between- D = ------------------------------- study [(sum wi)2 – sum (wi2)] variance) Q = sum wi(ln ORi – ln ORMH)2

  31. Random Effects: Meta-Analysis The addition of the between-variance term usually results in a more conservative estimate (larger confidence interval) than the fixed effects method. Larger studies AND those with disparate results are given more weight in the analysis. The inference applies to the research question at large, not just the studies included in the meta-analysis.

  32. Fixed Effect Note differences in Relative Weights Random Effects Note wider CI compared to fixed effect Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Research Synthesis Methods 2010; 1:97-111.

  33. SECTION 7.4 Heterogeneity among studies in meta-analyses

  34. Learning Outcome: Explain the importance of investigating heterogeneity among studies included in a meta-analysis.

  35. Heterogeneity Among Studies • Statistical heterogeneity: Disparate results across studies comprising a common research question. • Statistical question is whether there is greater variation between the results of the studies than is compatible with the play of chance. • Tests of heterogeneity typically lack statistical power – hence they are often inconclusive.

  36. More Recent Issues - Heterogeneity Heterogeneity: Petiti (2001) Approaches to heterogeneity in meta-analysis Higgins (2002) Quantifying heterogeneity in a meta-analysis Huedo-Medina (2006) Assessing heterogeneity in meta-analysis Lau (1998) Summing up evidence: one answer is not always enough

  37. Investigating Heterogeneity • Qualitative: Compare effect estimates of component studies by respective design and methodological characteristics: • • Year of publication • • Type/dose of intervention • • Outcome assessment • • Length and number of follow-up intervals • • Study inclusion/exclusion criteria • • Study size

  38. Investigating Heterogeneity • Quantitative: Investigate variation in effect estimates by statistical means: • • “Meta” regression analysis • --- variation in parameter of interest serves as the dependent variable • --- treatment intervention or factor of interest is included in the model with other study variables suspected to contributing of heterogeneity

  39. Scenario:Assume among 10 randomized • controlled trials, there is evidence of • statistical heterogeneity. The trials have • the following features: • All address the same basic research question • (e.g. is treatment A better than treatment B?) • They all have sufficiently large sample size. • They are all of reasonably good quality • (e.g. no obvious problems of validity). • What are some of the possible sources of • Heterogeneity seen between the 10 trials?

  40. Patient Population Heterogeneity: • • Age, gender, race, etc. • • Baseline medical history • • Baseline inclusion/exclusion criteria • Intervention Heterogeneity: • • Therapeutic improvements over time • • Individual versus group therapy

  41. Outcome Measurement • Heterogeneity: • • Study endpoints ascertained using • different criteria/measures • 4. Study Protocol Heterogeneity: • • Requirements for patient follow-up • (e.g. length, number of protocol • specified visits)

  42. Left Main LCx LAD Right Coronary Artery

  43. Testing for Statistical Heterogeneity • Q Test (Cochran, 1954) • Sum squared deviations of each study effect estimate from overall effect estimate, weighting contribution of each study by inverse of variance • H0: Homogeneity of effect sizes; Q statistic follows χ2 • distribution with k-1 (#studies) degrees of freedom • Q statistic poor power to detect true heterogeneity among studies when meta-analysis includes small number of studies. • Non-significant result is often inconclusive Q = wi(Ti – T)2

  44. Testing for Statistical Heterogeneity Q = wi(Ti – T)2 For Odds ratio, the effect size (T), is in the log scale – ln(OR) OR = (Odds of exposure among cases / Odds of exposure among controls) Ln(OR) = natural logarithm of odds ratio Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4)) Weight (W) = 1 / variance sum(weighti x ORi) Summary OR(T) = ---------------------------- sum(weighti) Critical value for χ2 at p=0.05= 9.49 (5-1 = 4 d.f. = see table 3 of textbook) Even though χ2 of 8.31 < 9.49, individual OR’s look different…….

  45. Practice: Testing for Statistical Heterogeneity Q = wi(Ti – T)2 For Odds ratio, the effect size (T), is in the log scale – ln(OR) OR = (Odds of exposure among cases / Odds of exposure among controls) Ln(OR) = natural logarithm of odds ratio Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4)) Weight (W) = 1 / variance sum(weighti x ORi) Summary OR(T) = -------------------------- = ------- = sum(weighti) Q = _______ Critical value for χ2 at p=0.05= _________ Conclusion: ________________________________

  46. Practice: Testing for Statistical Heterogeneity Q = wi(Ti – T)2 For Odds ratio, the effect size (T), is in the log scale – ln(OR) OR = (Odds of exposure among cases / Odds of exposure among controls) Ln(OR) = natural logarithm of odds ratio Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4)) Weight (W) = 1 / variance sum(weighti x ORi) 41.25 Summary OR(T) = -------------------------- = ------- = 1.03 sum(weighti) 39.99 Q = 6.13 Critical value for χ2 at p=0.05= 7.81 (4-1 = 4 d.f. = see table 3 of textbook) χ2 of 6.13 < 7.81 Conclusion: Do not conclude the studies differ in effect size.

  47. Testing for Statistical Heterogeneity • I2 Index (Higgins and Thompson, 2002) • Percentage of total variability in a set of effect sizes due to true heterogeneity (i.e. between-study variability) • Example: I2= 50 means half of total variability in effect sizes is not due to sampling error, but by true heterogeneity between studies. • classification of I2values (Higgins and Thompson, 2002) • I2 ~ 25%: “Low” heterogeneity • I2 ~ 50%: “Medium” heterogeneity • I2 ~ 75%: “High” heterogeneity I2 = ((Q - (n-1)) / Q) x 100 where n = number of studies (i.e. d.f for Q)

More Related