Statistical Analysis

1. Statistical Analysis MBS-01

2. Nominal Data • Dichotomous data • Categorizes variables • Assigns names, letters or descriptors • Gender, race, yes or no • No rank or mathematical relationship to each other • Have equivalent weight or value

3. Ordinal Data • Reflects an order to variables or data • Does not imply magnitude • Zero point is arbitrary • Pain Scale 0-10

4. Interval or Ratio Data Can compare absolute magnitude Implies a numeric relationship Can undergo arithmetic operations Data can be averaged and manipulated Continuous Data

5. Incidence Rate Proportion of group initially healthy that will develop a disease within a specified period of time Expressed in person-days, person-months, etc. Measures only new cases Prevalence Proportion of people in the population with disease at a given time Measures all existing cases Underestimates acute or rapidly occurring illnesses Measures of Disease Frequency

6. Relative Risk and Odds Ratio • Measures of association • Measures of disease frequency • Expressed as a single value • Describes the strength of association between the exposure and outcome • Does not imply any extent of variation • Often accompanied by a confidence interval

7. Relative Risk • How many times more likely an outcome is for one group compared with another • Ranges from 0 to infinity • RR of 0 is no association • RR of 1 is risk of acquiring disease same for subjects with and without risk factor • Association is stronger as RR increases (>10 felt to be strong association) • RR = 0.5 then initial risk is cut in half • RR = 2 then initial risk is doubled • Used in cohort (follow-up) design

8. Odds Ratio • Estimator of relative risk • Compares the prevalence of a disease when a specific factor is present or absent • Assumes cases & control gp representative of general population with respect to occurrence of risk factors • Assumes the frequency of disease in exposed or unexposed is small

9. Odds Ratio • Cross-sectional & Case-control design • Uses single value to describe strength of the association between exposure and outcome • OR <1 then risk as decreased • OR = 1 no association between risk factor and disease • OR >1 then risk has increased

10. Number Needed to Treat(NNT) • How many patients must be treated to get one good event (or prevent one bad event) • Applicable to groups of patients with similar underlying risk • Calculated from follow-up and experimental design studies

11. Relative Risk = A /(A+B) C/(C+D) Odds Ratio = A * D B * C Number Needed to Treat _________1___________ [A/(A+B) ] - [C/(C+D)]

12. Research vs Null Hypothesis Research Hypothesis: • The hypothesis tested by the study • Can be one tailed:a difference in only 1 direction • Can be two tailed:a difference in two directions Null Hypothesis: • Opposite of the research hypothesis • Hypothesis of no difference • Statistics are applied to the null hypothesis

13. Interval and Ratio Data Normal Distribution Parametric Nominal and Ordinal Data Nonnormal Distribution Non-Parametric

14. Independent Measurements • Seen in parallel design trials • Data do not depend on each other • Data do not reflect serial measurements Measure Outcome Variables Treatment Population Sample X Measure Outcome Variables Treatment

15. Dependent Measurements • Seen in cross-over design studies • Seen in studies using matched groups • Data depend on or reflect each other Sample Measure Outcome Measure Outcome Measure Outcome X Treatment

16. Statistical Analysis

17. Errors Related to Hypothesis

18. Alpha • The probability of making a Type I error • Predetermined by the investigator • Usual values 0.05 or 0.1 (1 in 10 or 1 in 20 chance of Type I error) • P value: the numeric representation of 

19. Beta and Power • Power is the probability of avoiding a Type II error. • Or the chance of finding a difference if it truly exists • Power is 1- • Increase power by increasing n, increasing , or increasing the size of difference accepted

20. Summary • Statistics tell us about the role that sampling variability plays in results • Statistics make no claim about the validity of a study • Consider the impact of Type I and II Errors • Results May be Statistically Significant but Clinically irrelevant.

21. Error, Validity and Precision

22. Random Error • Not constant error • Due to chance • Unknown sources of variation equally likely to affect findings in either direction • Seen as inconsistency in repeated or equivalent measurements when made on the same object or person • Increase sample size to reduce random error

23. Systematic Error • Constant error • Due to bias • Sources of variation that affect findings in one direction • Improve study design to reduce • Investigator should include explanation of systematic error in publication • Change of sample size will not affect systematic error

24. Reliability vs Validity of Data • Reliability: reproducibility of measurement • Validity: extent to which differences in scores reflect the true differences among individuals on the characteristic we are seeking to measure

25. Validity Study Results Internal Validity Truth in Study Results External Validity Errors of chance and bias Truth in the Universe

26. Threats to Internal Validity • History: naturally occurring event external to study but occurring simultaneously • Maturation: change in the study subject occurring as a function of the passage of time • Instrumentation: changes or errors in the measuring instrument or observer

27. Threats to Internal Validity • Selection: the way in which the subjects were selected and assigned to treatment groups • Experimental Mortality: dropout, nonresponse, or death • Main testing effect: being tested may bring about a change in behavior on a second observation • Statistical regression: tendency of extremes to move toward the mean during an experiment

29. Threats to External Validity • Interaction of Subject Variables and Tx: tx has varied effects on subgroups • Interactive Effect of Testing: pretesting may sensitize subjects to variable • Reactive Effects of Experimental Arrangements: study setting may be atypical • Multiple Treatment Interferences: same subject given several treatments; effects of earlier treatments not completely erasable • Hawthorne Effect: volunteers try to give “right” answer

30. Measures of Central Tendency • Mean: average • Median: midpoint where ½ of observations fall above and ½ fall below the value • Mode: most frequently encountered number • In a normal, or Gaussian, distribution the mean, median, and mode are identical

31. Precision • How closely the estimates will tend to cluster about the true value • Larger the standard deviation or standard error of the mean the less precise the data

32. Normality Mean, median, & mode

33. Skewed Distribution Mode Median Mean

34. Standard Deviation Measures how close the values cluster to the sample mean Interval Data Square root of variance Reported as +/- If normal distribution 1 S.D. = 68% data 2 S.D. = 95% data 3 S.D. = 99% data Standard Error of the Mean Estimates mean of population from sample meanˉх Equals S.D./ square root of n Smaller number than SD therefore often reported as measure of dispersion Measures of Variability

35. Confidence Intervals • Further defines the p value by giving a range of values to describe the data • An interval that will, with the probability of a confidence level, contain the true difference being investigated • A confidence interval which includes “0” does not permit rejection of the null

36. Sample Size, Meta-Analysis & Evidence Based Medicine MBS-01

37. Sample Size • Determined before initiation of study • Re-evaluated at conclusion of study due tp dropouts or deaths • Often not included in publication • Nomograms, formulas, tables are available to assist reader with sample size calculation • Studies of inadequate sample size: pilot studies

38. Sample Size Determinants • The outcome being evaluated • Tolerable risk of error (α and β) • Clinically important difference () • Variability of measurement (s and s2) • Ratio of experimental to control subjects

39. Sample Size Determinants • The outcome being evaluated • Dichotomous outcome • Continuous outcome • Tolerable risk of error (α and β) • Clinically important difference () • Variability of measurement (s and s2) • Ratio of experimental to control subjects

40. Sample Size Determinants • The outcome being evaluated • Tolerable risk of error (α and β) • Type I (α) • concludes there is a difference when in fact there is not • convention sets risk at 1-in-20 chance or an α of 0.05 • Type II (β) • concludes there is no difference when in fact one exists • Convention sets risk at 2-in-10 chance or a β of 0.2 • Clinically important difference () • Variability of measurement (s and s2) • Ratio of experimental to control subjects

41. Sample Size Determinants • The outcome being evaluated • Tolerable risk of error (α and β) • Difference between experimental and control group () • What is clinically important • Detecting a small difference between groups requires larger sample size • Variability of measurement (s and s2) • Ratio of experimental to control subjects

42. Sample Size Determinants • The outcome being evaluated • Tolerable risk of error (α and β) • Clinically important difference () • Variability of measurement (s and s2) • Expressed as s for continuous variables • Not specified for dichotomous variables • Smaller spread around mean requires smaller sample size • Ratio of experimental to control subjects

43. Sample Size Determinants • The outcome being evaluated • Tolerable risk of error (α and β) • Clinically important difference () • Variability of measurement (s and s2) • Ratio of experimental to control subjects • One-to-one ratio minimizes sample size • Assumed to be one-to-one in Young Nomogram

44. Sample Size • Equations & nomograms differ for type of outcome measured; nominal vs continuous • Z (Standard Normal) distribution is used for alpha and beta • Mean of 0 • S.D. of 1 • Can solve for any portion of equations if other factors are known • Plan for drop-out

45. Sample Size Equation for Dichotomous Variables P: proportion of responders in both groups P1: proportion of responders in experimental group P0: proportion of responders in control groups Use 2 sided alpha of 0.05  Zα =1.96 Use one-sided beta of 0.20  Zβ = 0.84

46. Sample Size Equation for Continuous Variables Use 2 sided alpha of 0.05  Zα =1.96 Use one-sided beta of 0.20  Zβ = 0.84  : difference in means for control & experimental group σ: weighted average standard deviation in the control group

47. Nomograms for Calculation of Sample Size • Assumes parallel study (no crossover) with two groups • Assumes 2 tailed alpha; if not, overestimates needed sample size • Assumes alpha 0.05 and beta 0.2 • Approximation used retrospectively to critique another author’s conclusions

48. Number of Patients Nomogram

49. MBS-01

50. Sample Size and Study Outcome • Difference Observed • n is large enough to identify difference • Difference may be overestimated if n is smaller than would have been determined using sample size calculations • Extremely large n may find difference but clinical significance of the difference lacking • No difference Observed • n may be too small • Truly may be no difference