Section 9.1(re-visited)

1. Section 9.1(re-visited) Making Sense of Statistical Significance Inference as Decision

2. Warm-up • The one-sample t statistic for testing H0: μ = 0 and Ha: μ > 0 from a sample of n = 15 observations has the value t = 1.82 • What are the degrees of freedom for this statistic? • Between what two values does the P-value of the test fall? • Is the value t = 1.82 significant at the 5% level? Is it significant at the 1% level?

3. Practical Applications • In practice, statistical tests are used for marketing, research, and the pharmaceutical industry. • The decisions we make as statisticians must have practical significance. This means that it must be worthwhile to use the information we find significant.

4. Points to Keep in Mind • If you are going to make a decision based on a statistical test, choose α in advance. • When choosing α, ask these questions: • Does H0 represent an assumption that people have believed for years? If so, then strong evidence (small α) is needed to persuade them. • What are the consequences of rejecting H0? Costly changes will require strong evidence.

5. Statistical Significance vs.Practical Significance • Statistical significance is based on the hypothesis test. • A large sample size will almost always show that small deviations are significant. • Why? • Practical significance means the data isn’t convincing enough to make a change.

6. Example of statistical significance that is not practical • Suppose we are testing a new antibacterial cream, “Formulation NS” on a small cut made on the inner forearm. We know from previous research that with n medication, the mean healing time (defined as the time for the scab to fall off) is 7.6 days, with a standard deviation of 1.4 days. The claim we want to test here is that Formulation NS speeds healing. We will use a 5% significance level. • We cut 25 volunteer college students and apply Formulation NS to the wound. The mean healing time for these subjects is x-bar = 7.1 days. We will assume that σ = 1.4 days.

7. Solution • We find that the data is statistically significant. • However, it does not appear that the effect is all that great. Is it practical to use this treatment if it only reduces the amount of time you have a scab by about a day?

8. Cautions PROCEEDWITH….. • Watch out for badly designed surveys or experiments! • Statistical inference cannot correct for basic flaws in design. • Always plot the data (if it’s given to you) and look for outliers or other deviations from a consistent pattern.

9. Type I and Type II Errors • Sometimes our decision (reject or fail to reject H0) will be wrong. • We could reject H0 when we shouldn’t have. • We could fail to reject H0 when we should have.

10. In words… • Type I Error: Reject H0 when H0 is actually true. • Type II Error: Fail to reject H0 when H0 is actually false.

11. Why do we care about errors? • If a potato chip factory rejects bags of chips that statistically fail to meet a salt value, they lose money if the batch is really ok. • On the other hand, if they fail to reject a batch that has too much salt, they will have unhappy customers. Type I Type II

12. Probability of Type I error • The probability of a Type I error occurring is equal to alpha.

13. The state of Georgia is considering spending $350 million on a computerized mathematics curriculum that for grades 3 –10. The pilot the program is with 250 students in grades 5 and 6 whose end of course test scores are compared to the state average score of 150. Will this improve test scores in the state of Georgia?

14. Find Probability of Type I error • The mean salt content of a certain type of potato chips is supposed to be 2.0mg. The salt content of these chips varies normally with standard deviation σ = 0.1mg. From each batch produced, an inspector takes a sample of 50 chips and measures the salt content of each chip. The inspector rejects the entire batch if the sample mean salt content is significantly different from 2mg at the 5% significance level.

15. I’ve Got the Power! • Power is good! • Power is the probability that a fixed α level significance test will reject H0 if Ha is true. • Power of a test = 1 – P(Type II Error) • Power can increase by having a larger n. More and more often, statisticians are looking at the power of a study along with confidence intervals and significance tests

16. Error Probabilities The potato-chip producer wonders whether the significance test of H0: p = 0.08 versus Ha: p > 0.08 based on a random sample of 500 potatoes has enough power to detect a shipment with, say, 11% blemished potatoes. In this case, a particular Type II error is to fail to reject H0: p = 0.08 when p = 0.11. Significance Tests: The Basics What if p = 0.11? Power and Type II Error The power of a test against any alternative is 1 minus the probability of a Type II error for that alternative; that is, power = 1 - β. Earlier, we decided to reject H0at α = 0.05 if our sample yielded a sample proportion to the right of the green line. Since we reject H0at α= 0.05 if our sample yields a proportion > 0.0999, we’d correctly reject the shipment about 75% of the time.

17. Homework Chapter 9 # 19 - 23