More About Tests and Intervals

1. More About Tests and Intervals Ch. 21

2. How to Think About P-Values • The conditional probability of the observed statistic given that the null hypothesis is true. • NOT the probability that the null hypothesis is true. • NOT the conditional probability that the null hypothesis is true given the data

3. Alpha Level Alpha Level: • The threshold for determining when an event is too rare • Also called the significance level

4. Statistical Significance • When the P-value falls below the alpha level, we say the result is “statistically significant” • Statistically significant means not likely due to random variation

5. Sometimes We Screw Up • The data can lead us astray in two ways: • The null hypothesis is true, but we mistakenly reject it. (Type I error) • The null hypothesis is false, but we fail to reject it. (Type II error)

6. Screwing Up • Which type of error is more serious depends on the situation at hand. In other words, the gravity of the error is context dependent.

7. Type I Error • The null hypothesis is true, but we mistakenly reject it. • The probability of a Type I error is the Alpha () level

8. Type II Error • The null hypothesis is false, but we fail to reject it. •  is the probability of a Type II error • We don’t know 

9. Getting it Right • When H0 is false and we reject it, we have done the right thing. • A test’s ability to detect a false hypothesis is called the power of the test. • Power of the test:

10. A Picture Worth Words • This diagram shows the relationship between Type I error, Type II error, and Power