What the clinician needs to know about statistics : when is biostatistics not to be trusted ?

What the clinicianneedstoknowaboutstatistics: whenisbiostatisticsnottobetrusted? Giuseppe Biondi Zoccai DivisionofCardiology, UniversityofTurin, Turin, Italy Meta-analysis and Evidence-based medicine Training in Cardiology(METCARDIO), Ospedaletti, Italy 2nd Fellows’ Meeting, 2-3 October 2009, Bubbio – 3 October 2009, 10:30-10:45

LEARNING GOALS • What are the goals of biostatistics in clinical research? • Is there a hierarchy in biostatistics? • When is biostatistics not to be trusted?

GOALS OF BIOSTATISTICS • Biostatistics is mainly used for causality appraisal in clinical research • Biostatistics alone cannot however enable causality inference (i.e. necessary but not sufficient) • As it works within Popper’s approach, it can only ultimately falsify and never actually prove

KARL POPPER’S EPISTEMOLOGY • You can never prove thatsomethingiscorrect in science, you can onlydisprovesomething, i.e. show itis wrong • Thus, onlyfalsifiablehypotheses are scientific* • (and that’s whyreligionisnotscientific)

BRADFORD HILL CRITERIA FOR CAUSATION • Strength:*precisely (p<0.05) defined and strong relative risk (≤0.83 or ≥1.20) in the absenceofmultiplicityissues (strongest) • Consistency:*findingofassociationneedstobereplicated in otherstudies • Temporality: toinfercausality, exposuremustappropriately precede outcome • Coherence: makescause-effectrelationshipshouldnotconflictwithnaturalhistory/biologicfacts *STATISTICS PLAYS A ROLE HERE Mente et al, Arch Intern Med 2009

BRADFORD HILL CRITERIA FOR CAUSATION • Biological gradient:*dose of exposure and risk of disease are positively (or negatively) related • Experiment: experimental evidence from laboratory studies (weak) or randomized clinical trials (strongest) • Specificity: specific exposure is related to 1 disease only (NA for multifactorial diseases) • Plausibility: makes biological/clinical sense (weak) • Analogy: arguing on analogical reasoning (weakest) *STATISTICS PLAYS A ROLE HERE Mente et al, Arch Intern Med 2009

EBM HIERARCHY OF EVIDENCE • N of 1 randomizedcontrolled trial • Systematicreviewsofhomogeneousrandomizedtrials • Single randomized trial • Systematicreviewofobservationalstudiesaddressingpatient-importantoutcomes • Single observationalstudyaddressingpatient-importantoutcomes • Physiologicstudies(egbloodpressure, cardiac output, exercisecapacity, bone density, and so forth) • Unsystematicclinicalobservations Guyatt and Rennie, Users’ guide to the medical literature, 2002

PARALLEL HIERARCHY OF RESEARCH MORE FLEXIBLE BUT LESS VALID Qualitative reviews Case reports and series Observational studies Systematic reviews Observational controlled studies Meta-analysesfromindividualstudies Single center randomizedcontrolledtrials Meta-analyses from individual patient data Multicenter randomized controlled trials LESS FLEXIBLE BUT MORE VALID Biondi-Zoccai et al, ItalHeart J 2003

IS A RANDOMIZED TRIAL ALWAYS NEEDED? WE WERE UNABLE TO IDENTIFY ANY RCT OF PARACHUTE INTERVENTION FOR GRAVITATIONAL CHALLENGE… WE THINK THAT EVERYONE MIGHT BENEFIT IF THE MOST RADICAL PROTAGONISTS OF EBM ORGANIZED AND PARTICIPATED IN A DOUBLE BLIND, RANDOMIZED, PLACEBO CONTROLLED TRIAL OF PARACHUTE INTERVENTION Smith et al, BMJ 2003

THE INAPPROPRIATE USE OF A FANCY STATISTICAL TOOL PROPENSITY SCORES ARE USELESS IN LARGE STUDIES WITH AN ADEQUATE (>7) NUMBER OF EVENTS PER COVARIATE Cepeda et al, Am J Epidemiol 2003

THE WEAK STATISTICAL EVIDENCE DO YOU TRUST A CONFIDENCE INTERVAL REACHING 1.01 (OR A P=0.049)? Hannan et al, Circulation 2006

THE LARGE CONFIDENCE INTERVAL CONFIDENCE INTERVAL SPANNING FROM 0.15 TO 0.97 Patti et al, Circulation 2005

THE UNREASONABLY COMPLEX METHOD ? Huyhn et al, Circulation 2009

THE UNREASONABLY COMPLEX METHOD ? …WE SELECTED NONINFORMATIVE PRIOR DISTRIBUTIONS… THESE INCLUDED NORMAL DENSITIES (MEAN, 0; 0.00001 [VARIANCE OF 105]) FOR THE LOGARITHM OF THE ORS AND (UNIFORM ON THE INTERVAL [0,2]). TO ENSURE CONVERGENCE OF THE GIBBS SAMPLER ALGORITHM, 3 MARKOV MONTE CARLO CHAINS WERE RUN, AND CONVERGENCE WAS ASSESSED AFTER 60000 ITERATIONS. THE FINAL STATISTICS WERE BASED ON 120000 ITERATIONS, 100000 OF THEM FOR BURN-IN. Huyhn et al, Circulation 2009

TAKE HOME MESSAGES

TAKE HOME MESSAGES • Biostatistics is merely necessary, but not sufficient alone to infer causality • Biostatistical comparisons can really be trusted only if stemming from one or more randomized trials (e.g. in a meta-analysis) • Sophisticated statistics cannot remedy faulty study designs, fabricated or missing data • Sophisticated statistics can cloud a manuscript weaknesses, ominously separating clinicians from decision-making

Thank you for your attentionFor any correspondence: gbiondizoccai@gmail.comFor these and further slides on these topics feel free to visit the metcardio.org website:http://www.metcardio.org/slides.html

What the clinician needs to know about statistics : when is biostatistics not to be trusted ?