1 / 36

Sample Size And Power Warren Browner and Stephen Hulley

Sample Size And Power Warren Browner and Stephen Hulley. The ingredients for sample size planning, and how to design them An example, with strategies for minimizing sample size. Sampling and Inference. A sample is designed to represent a larger population

branden-fitzpatrick
Download Presentation

Sample Size And Power Warren Browner and Stephen Hulley

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sample Size And PowerWarren Brownerand Stephen Hulley • The ingredients for sample size planning, and how to design them • An example, with strategies for minimizing sample size

  2. Sampling and Inference • A sample is designed to represent a larger population • Therefore, findings in the sample allow inferences about events in the population • Problem: what if the inferences are wrong? • Finding something in the sample that isn’t “real” in the population • Missing something that is “real”

  3. Preventing Wrong Inferences • Difficult when caused by systematic error (bias) • Easy when caused by Random error (chance) • Solution: increase sample size • Problem: cost, feasibility • Goldilocks solution: a sample size that is big enough but not too big

  4. Ingredients For Planning Sample Size in an Analytic Study or RCT • Hypothesis • Null and alternative • One-sided vs two-sided • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  5. Research Hypothesis • A clear statement of what you are studying • Simple: one predictor, one outcome • Specific: who, what, when, where • Stated: in advance

  6. Research Hypothesis • In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with newmol will have a lower 1-year mortality than those randomly assigned to placebo.

  7. The Null Hypothesis • There’s nothing going on. • Purpose in life: to be rejected in favor of its alternative. • In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with newmol will have the same 1-year mortality as those randomly assigned to placebo.

  8. What’s This All About? • A long time ago, statisticians figured out the probability that a sample of a given size would “find something” even if there were nothing going on in the population.

  9. This means that... • After a study, we can determine the likelihood that whatever we found in our sample could have occurred by chance... • Even if nothing was going on in the population (i.e., the null hypothesis was true)--a “Type I error” • If this is very unlikely (say < 1 in 20) we reject the null hypothesis in favor of the alternative hypothesis; we call the finding statistically significant (P < .05)

  10. Two-sided Alternative Hypothesis • In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with newmol will have a different1-year mortality than those randomly assigned to placebo.

  11. Two One-sided Alternative Hypotheses • Side A: In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with newmolwill have a higher 1-year mortality than those randomly assigned to placebo. • Side B: In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with newmol will have a lower 1-year mortality than those randomly assigned to placebo.

  12. If The Null Hypothesis Is True • By chance alone, each of the two one-sided alternative hypotheses is... • Possible • Equally likely • Wrong • Thus a two-sided alternative hypothesis has twice the likelihood of happening by chance alone

  13. Next Ingredient: Statistical Test (Types of Variable) • The statistical test determines how the sample size will be calculated • The type of predictor and outcome variable determine which statistical test will be used to analyze the data • Both dichotomous: Chi square • One dichotomous, one “continuous”: t test • Both “continuous”: correlation coeff or t test

  14. Statistical Test (Types of Variable) • ALS study • Predictor: newmol vs placebo • Outcome: % dead • Both are dichotomous • Chi square test

  15. Next Ingredient:Effect Sizes (dichotomous variables) • How big an effect you anticipate seeing • Newmol halves mortality • newmol = 5%, Placebo = 10%

  16. Penultimate Ingredient: Power • The chance of finding something in your sample if it’s really going on in the population (avoiding a Type II error) • “Something” = the effect size (or greater) • Usually set at 80% or 90% • = (1 - beta)

  17. …and the Final Ingredient: Alpha • The chance of finding something in your sample if there’s nothing going on in the population.

  18. Alpha Explained • The level of statistical significance (ie, the p-value that will be considered significant) • The pre-set maximum chance of finding something, if it really isn’t there. • Usually set at 0.05. • May be one-sided or two-sided.

  19. Sidedness Of Alpha • With a two-sided alternative hypothesis, you have two chances of finding something that isn’t really there: • One (equal) chance for each side. • So a one-sided alpha of 0.05 corresponds to a two-sided alpha of 0.10.

  20. SAMPLE SIZE: AN EXAMPLE • Null hypothesis: • In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with newmol will have the same 1-year mortality as those randomly assigned to placebo. • Two-sided alternative hypothesis • Dichotomous predictor and outcome • Effect size: 10% mortality + 5% • Power, alpha: 90%, 0.05 (two-sided)

  21. THE SAMPLE SIZE IS… • Appendix 6.B • Smaller of P1 and P2 = 0.05; power of 90%; alpha of 0.05 (two-sided) • Difference = 0.05 381 473 620 • This is per group

  22. Sample Size Reduction Strategy #1:Statistical Manipulation • Use a lower power • Use a one-sided alpha • Power of 80% • One-sided alpha of 0.05

  23. The New Sample Size Is… • Appendix 6.B • Smaller of P1 and P2 = 0.05; power of 80%; alpha of 0.05 (one-sided) • Difference = 0.05 381 473 620 • This is also per group

  24. SS Reduction Strategy #2: Use A More Common Outcome • Change from 1-year mortality to 2-year mortality or loss of independent living • Placebo: 40% • Newmol: 20%

  25. The New Sample Size Is… • Appendix 6.B • Smaller of P1 and P2 = 0.20; power of 80%; alpha of 0.05 (two-sided) • Difference = 0.20 74 91 118

  26. SS Reduction Strategy #3: Use A Continuous Outcome • Change “mortality or loss of independent living” to “muscle strength” • NOTE: Big change in research question and research hypothesis. • New null hypothesis: • In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with newmol will have the same grip strength at the end of six months as those treated with placebo. • Two-sided alternative hypothesis

  27. Estimate The Mean And Variability Of Grip Strength • Patients with untreated ALS have a (mean ± SD) grip strength of 20 ± 10 kg after 6 months of disease • Newmol may improve that by 25%

  28. Then • Grip strength • Placebo: 20 kg • Newmol: 25 kg (25% more) • Effect size = 5 kg SD = 10 kg • Standardized effect size: E/S = 5/10 = 0.5

  29. The New Sample Size Is ... • Appendix 6.A • E/S = 0.5 • ß = 0.20, Alpha (two-sided) = 0.05 • N = 64per group

  30. Ss Reduction Strategy #4: Use A More Precise Outcome • Buy a better instrument to measure grip strength • Use a well-defined protocol • Repeat measurements on two consecutive days • Reduce SD from 10 kg to 8 kg

  31. The New Sample Size Is ... • New E/S = 5 kg/8 kg= 0.625 • ß = 0.20, Alpha (two-sided) = 0.05 • N = about 45per group • This helped quite a bit.

  32. SS Reduction Strategy #5: Use Paired Measurements • Most of the variability in grip strength at the end of the study is likely to be due to differences between subjects in grip strength at the beginning of the study. • Switch the outcome to change in grip strength from the beginning to the end of the study.

  33. Paired Measurements • Each subject contributes a pair of measurements: (before, after) • The outcome variable is the difference between that pair for each subject. • The SD of the change in a measurement is usually < than the SD of the measurement • SD of change in grip strength is 5 kg • New standardized effect size = 5/5 = 1.0

  34. The New Sample Size Is... • E/S = 1.0 • ß = 0.20, Alpha (two-sided) = 0.05 • N = 17per group • We now have a potentially do-able study, albeit one that is very different from the original aim.

  35. The Bottom Line • Sample size estimation is an integral part of study planning • Almost never the last thing you do • More often, one of your first tasks

  36. SAMPLE SIZE PLANNING: REVIEW OF INGREDIENTS • Looking for something in a sample • Hypotheses (null and alternative) • Will you be able to ... • Know it’s there in the population if you find it in your sample (avoid a Type I error) • Test of significance, alpha • Find it in your sample if it’s there in the population (avoid a type II error)? • Effect size, power

More Related