1 / 21

Effect Size & Power Analysis

Effect Size & Power Analysis. 0246-511 Statistics for Graduate Study I. Learning Objectives. Understand common effect size measures Which to use When to use different types Learn to employ effect sizes to plan study A priori power analysis Two groups ANOVA Observed power. Effect Size.

mfrey
Download Presentation

Effect Size & Power Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Effect Size & Power Analysis 0246-511 Statistics for Graduate Study I

  2. Learning Objectives • Understand common effect size measures • Which to use • When to use different types • Learn to employ effect sizes to plan study • A priori power analysis • Two groups • ANOVA • Observed power

  3. Effect Size • Two Broad Types (in ANOVA) • Standardized mean differences • Proportion of variance accounted for • Compared to statistical significance • APA Guidelines • Use in Meta-Analysis

  4. Effect Size: Comparing Two Means Standardized Mean Differences Population Value Estimated from sample Proportion of Variance Accounted for

  5. Concepts (Two-Groups) • Power • Type I & Type II Error • Effect Size • γ (gamma) • δ (delta) • Estimating Power • Estimating Needed Sample Size • Observed Power

  6. Review • Power • Type I Error • Type II Error • Factors that affect power

  7. Independent Samples t-test:Things we need to estimate power • Estimate Effect Size • Estimate Gamma • Estimate Delta • Estimate Power • Working backwards to get N

  8. Fire Safety Instruction Two methods of teaching Pilot Study Need to estimate sample size needs to get funding Power of .80 at alpha .05 and .01 An example

  9. Estimate of Effect Size Gamma Estimate Delta Estimate (observed) Power Work backwards to get n We will need to

  10. Results from Pilot Study Observed Power (.05) ≈ 0.32 Observed Power (.01) ≈ 0.14 Note: strictly speaking, all we need to estimate n for power of .80 is gamma, we compute observed power for illustration and because it is sometimes helpful.

  11. Obtaining estimate of n Note1: these are not obtained delta estimates, but based on what is needed to obtain desired power level Note 2: these are “n” values not “N” values, thus need this many per condition.

  12. Unequal Sample Sizes For unequal sample sizes we often use something called a harmonic mean. It is calculated as follows: If our group sizes had been 14 and 9:

  13. Effect Sizes: More than two means Population Based Measure Estimated from Sample Another sample measure Also called R2

  14. A brief digression ω2 can be derived from the F statistic as well: You will care shortly. 

  15. Cohen’s Effect Size Estimate • Cohen defines a population ANOVA ES estimate as: • Stevens provides an estimate of this as: • Keppel & Wickens provide another estimate:

  16. Extend previous example

  17. Use power chart to obtain a noncentrality paramter: Φ

  18. Using power charts to get sample size What is a noncentrality parameter? From examining a power chart, Φ≈1.8 Now we substitute our obtained Φ value into the following equation: Rounding up to 51, this implies 3(51-1)=150 for dfdenom. We need to iterate again. Using the infinity line, we get…

  19. Strategy for assessing power • Determine form of experiment/research • Decide on hypothesis test • Carefully consider which effect(s) is (are) important to detect • Select desired power (& alpha) • Determine likely effect size (or range of effect sizes) • Conduct power analysis • Consider whether study is feasible as currently designed

  20. In-Class Example • Students’ (MA Thesis) involves trying to find a hypothesized effect. • Wish to detect effect it if it is at least ω2=.03 • df numerator will be 1 • How many subjects will she have to recruit?

  21. Software Options • SPSS (observed power) • G*Power (consistent with Dr. Jackson’s 2nd principle of economics) • Web-based power calculators • Others

More Related