Power of a test

1. Power of a test

2. The power of a test (against a specific alternative value) • Is the probability that the test will reject the null hypothesis when the alternative is true • In practice, we carry out the test in hope of showing that the null hypothesis is false, so high power is important

3. Suppose H0 is false – what if we decide to reject it? Suppose H0 is false – what if we decide to fail to reject it? We correctly reject a false H0! Suppose H0 is true – what if we decide to fail to reject it? Type I Correct a Power Suppose H0 is true – what if we decide to reject it? Correct Type II b

4. What is power? • Power is the probability of rejecting the null hypothesis when in fact it is false. • Power is the probability of making a correct decision (to reject the null hypothesis) when the null hypothesis is false. • Power is the probability that a test of significance will pick up on an effect that is present. • Power is the probability that a test of significance will detect a deviation from the null hypothesis, should such a deviation exist. • Power is the probability of avoiding a Type II error.

5. What is power? • Power is the probability of rejecting the null hypothesis when in fact it is false. • Power is the probability of making a correct decision (to reject the null hypothesis) when the null hypothesis is false. • Power is the probability that a test of significance will pick up on an effect that is present. • Power is the probability that a test of significance will detect a deviation from the null hypothesis, should such a deviation exist. • Power is the probability of avoiding a Type II error.

6. A researcher selects a random sample of size 49 from a population with standard deviation s = 35 in order to test at the 1% significance level the hypothesis: H0: m = 680 Ha: m > 680 What is the probability of committing a Type I error? a = .01

7. H0: m = 680 Ha: m > 680 For what values of the sample mean would you reject the null hypothesis? Invnorm(.99,680,5) =691.63

8. H0: m = 680 Ha: m > 680 If H0 is rejected, suppose that ma is 700. What is the probability of committing a Type II error? What is the power of the test? Normalcdf(-10^99,691.63,700,5) =.0471 Power = 1 - .0471 = .9529

9. H0: m = 680 Ha: m > 680 If H0 is rejected, suppose that ma is 695. What is the probability of committing a Type II error? What is the power of the test? Normalcdf(-10^99,691.63,695,5) =.2502 Power = 1 - .2502 = .7498

10. ma Fail to Reject H0 Reject H0 a m0 Power = 1 -b b

11. What happens to a, b, & power when the sample size is increased? Fail to Reject H0 Reject H0 a m0 b ma

12. Facts: • The researcher is free to determine the value of a. • The experimenter cannot control b, since it is dependent on the alternate value. • The ideal situation is to have a as small as possible and power close to 1. (Power > .8) • Asa increases, power increases. (But also the chance of a type I error has increased!) • Best way to increase power, without increasing a, is to increase the sample size n • Decreasing  will increase power, but this is usually not feasible. • Another way to increase power is to consider an alternative farther from µ0

13. For the AP (and my) Exam • An understanding of error and power is what’s important, not the mindless calculations that most students don’t understand anyway. • STUDY the types of error, make sure you can state them in words. • Be sure to state what the power of the test IS and how to increase it! • ap central - on power

14. Bottles of a popular cola are suppose to contain 300 ml of cola. A consumer group believes the company is under-filling the bottles. (Assume s = 50 with n = 30) Find the power of this test against the alternative m = 296 ml.