STAT 3130 Statistical Methods II

1. STAT 3130Statistical Methods II Session 6 Statistical Power

2. STAT3130 – Statistical Power Ho TRUE Ho FALSE Reject Ho FTR Ho When we engage in Hypothesis Testing, we have focused on the p-value and the alpha value…recall what these actually mean.

3. STAT3130 – Statistical Power Ho TRUE Ho FALSE Reject Ho FTR Ho A second calculation of interest is the Power of the test – or 1- the probability of committing a Type 2 error. In words, the Power of the test is “The probability of correctly rejecting the Null Hypothesis when it should be rejected”.

4. STAT3130 – Statistical Power In other words, it is the probability that if a true difference exists, it will be discovered. It is sometimes referred to as the “sensitivity” of the test. Statistical Power is heavily used in medicine, clinical psychology and biology. Typically, a test must have a Statistical Power of 80% or greater to be considered valid. Why would that be?

5. STAT3130 – Statistical Power • Power is a function of three factors: • Effect size – i.e., the difference between the two groups or measurements. As the effect size goes up, the power increases. • Alpha – As the chance of finding an incorrect significant effect is reduced (Type I error), the probability of correctly finding an effect is also reduced. Typically, alpha is set to be .01 (most conservative and lowers power), .05 or .10 (most risk tolerant and increases power). As alpha goes down, the power increases. • Sample Size - Increased sample sizes will always produce greater power. But, increasing the sample size can also produce “too much” power – smaller and smaller effects will be found to be significant until at a large enough sample size, any effect is considered to be significant. As sample size goes up, power increases.

6. STAT3130 – Statistical Power When conducting a one tailed test, the power calculation is executed as: 1-() = 1-P[z<z- (| 0 - a|/ (s/SQRT(n))] A two tailed test is conducted similarly, with the alpha value associated with the z score divided by two.

7. STAT3130 – Statistical Power Example… Lets say that we have a treatment group taking a cholesterol lowering drug. Lets say that the average decrease in cholesterol after taking the drug is 2 points. (n=100 and s = 10). What do you conclude. 1-(2) = 1-P[z<1.645- (| 2|/ (10/SQRT(100))] Now…what if the drop is 3 points. What happens?

8. STAT3130 – Statistical Power Lets build some power curves… Power versus effect size Power versus sample size

9. STAT3130 – Statistical Power If you are starting an experiment from scratch, how do you determine the appropriate sample size? n= 2*(Z +Z )2/Δ2 A two tailed test is conducted similarly, with the alpha value associated with the z score divided by two. (Or…you can just use SAS).