1 / 37

Introduction to Power Analysis

Introduction to Power Analysis.  G. Quinn & M. Keough, 2003 Do not copy or distribute without permission of authors. Power of test. Probability of detecting an effect if it exists Probability of rejecting incorrect H O 1 – b, where b is the Type II error. Region where H o retained.

raja
Download Presentation

Introduction to Power Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Power Analysis G. Quinn & M. Keough, 2003 Do not copy or distribute without permission of authors.

  2. Power of test • Probability of detecting an effect if it exists • Probability of rejecting incorrect HO • 1 – b, where b is the Type II error

  3. Region where Ho retained Region where Ho rejected Ho HA Type II error Type I error

  4. Statistical power depends on • Effect size (ES) • size of difference between treatments • large effects easier to detect • Background variation: • variation between experimental units (s2 estimated by s2) • greater background variability, less likely to detect effects

  5. Sample size (n) for each treatment group: • increasing sample size makes effects easier to detect • Significance level (a): • Type I error rate • as a decreases, b increases, power decreases

  6. Power analysis If a is fixed (usually at 0.05), then

  7. Exact formula depends on statistical test (i.e. different for t, F etc.)

  8. c2 z P(y) t F P(y) y y

  9. Diminshing returns Good returns

  10. A posteriori power

  11. Post-hoc (a posteriori) power analysis • If conclusion is non-significant: • report power of experiment to detect relevant effect size. • Solve power equation for specific ES:

  12. Karban (1993) Ecology 74:9-19 • Plant growth and reproduction in response to reduced herbivores. • Two treatments: • normal herbivore damage • reduced herbivore damage • n = 31 plants in each treatment • For plant growth: F1,60 = 0.51, P = 0.48 ns

  13. Karban (1993) Ecology 74:9-19 Power to detect effects: Small effect (ES = 0.1) 0.11 Medium effect (ES = 0.25) 0.50 Large effect (ES = 0.40) 0.88 Effect size (ES) = ÖMSGroups / ÖMSResidual ie. SD Groups / SD Reps - see Cohen (1992)

  14. A priori power analysis sample size determination To determine appropriate sample size a priori, we need to know: • what power we want • background variation (from pilot study or previous literature) • what ES we wish to be able to detect if it occurs

  15. Solve power equation for n

  16. Required sample size

  17. Example of a priori power analysis • Effects of fish predation on mudflat crabs • Two treatments: • cage vs cage control • Pilot study: • number of crabs in 3 plots • variance was 19 (sos2 = 19) • mean was 20

  18. Aims: • to detect 50% increase in crab numbers due to caging, ie. an increase from 20 to about 30; so ES = 50% (or 10 crabs per plot) • to be 80% sure of picking up such an effect if it occurred; so power = 0.80 • How many replicate plots required for each treatment? • what is required n?

  19. Basic power (t-test)ES = 10 s = 4.36

  20. Detecting a more subtle effectHalved

  21. If data are more variableVariance doubled

  22. Minimum Detectable Effect Size • If an ES can’t be determined • Specify target power, solve for ES

  23. ES vs Sample Size(a priori)

  24. Effect size • How big: • what size of effect is biologically important? • how big an effect do we want to detect if it occurs?

  25. An effect size is…

  26. Effect size • Where from? • biological knowledge • previous work/literature • compliance requirements (e.g. water quality)

  27. Specification of effect size • Easy for 2 groups: • difference between 2 means Central t Noncentral t P(y) 0

  28. Specification of effect size • Harder for more than 2 groups: • Consider 4 groups: • 50% difference from smallest to largest • 1 = 2 = 3 < 4? • 1 < 2 < 3 < 4? • 1 = 2 < 3 = 4? • Shape of alternative distribution depends on the particular pattern

  29. Central F P(y)

  30. One-way ANOVARange = 10, s = 6.2 1.00 0.80 3 vs 1 0.60 Power 2 vs 2 0.40 Linear 0.20 0.00 2 4 6 8 10 No. of replicates

  31. Estimate of variance • Other work on same system • Published work on similar systems • Pilot studies • Must be estimate of same kind of variance • e.g. paired vs two-sample t test • Variance of difference vs variance of each sample

  32. Power analysis requires • Clear understanding of the kind of statistical model to be used (inc. the formal tests) • Careful thought about important effects; hardest step, especially for interactions • An estimate of variance • Significance level to be used • Desired level of confidence • Understanding of non-centrality parameters for complex designs

  33. Cautions • Variance estimates may be uncertain • Allow for extra samples in case of larger than expected variation • Realized power • Cohen’s Effect Sizes • Raw & standardized ES • Terminology

  34. Options for study planning: n • Level of significance • Desired power • Target effect size • Estimate of variation • Calculated sample size • or Power vs n • “Safety” factor

  35. Options for planning: ES • Significance level • Desired level of confidence • Estimate of variation • Suggested sample size (or range) • Effect that should be detected • ES vs n

  36. Power calculations • Charts & tables available in many books • Software will do these calculations: • Gpower • www.zoology.unimelb.edu.au/stats • PiFace & Java applets • www.uiowa.edu/~rlenth/power • Review at: • http://sustain.forestry.ubc.ca/cacb/power • Some statistical packages • But check what they do!

  37. Power vs ES(post hoc)

More Related