The Practice of Statistics, 4 th edition – For AP* STARNES, YATES, MOORE

1. Chapter 9: Testing a Claim Section 9.1 Significance Tests: The Basics The Practice of Statistics, 4th edition – For AP* STARNES, YATES, MOORE

2. Chapter 9Testing a Claim • 9.1 Significance Tests: The Basics • 9.2 Tests about a Population Proportion • 9.3 Tests about a Population Mean

3. Section 9.1Significance Tests: The Basics Learning Objectives After this section, you should be able to… • STATE correct hypotheses for a significance test about a population proportion or mean. • INTERPRET P-values in context. • INTERPRET a Type I error and a Type II error in context, and give the consequences of each. • DESCRIBE the relationship between the significance level of a test, P(Type II error), and power.

4. Significance Tests: The Basics • Introduction Confidence intervals are one of the two most common types of statistical inference. Use a confidence interval when your goal is to estimate a population parameter. The second common type of inference, called significance tests, has a different goal: to assess the evidence provided by data about some claim concerning a population. A significance test is a formal procedure for comparing observed data with a claim (also called a hypothesis) whose truth we want to assess. The claim is a statement about a parameter, like the population proportion p or the population mean µ. We express the results of a significance test in terms of a probability that measures how well the data and the claim agree.

5. Significance Tests: The Basics • The Reasoning of Significance Tests Statistical tests deal with claims about a population. Tests ask if sample data give good evidence against a claim. A test might say, “If we took many random samples and the claim were true, we would rarely get a result like this.” To get a numerical measure of how strong the sample evidence is, replace the vague term “rarely” by a probability. Suppose a basketball player claimed to be an 80% free-throw shooter. To test this claim, we have him attempt 50 free-throws. He makes 32 of them. His sample proportion of made shots is 32/50 = 0.64. What can we conclude about the claim based on this sample data? We can use software to simulate 400 sets of 50 shots assuming that the player is really an 80% shooter. You can say how strong the evidence against the player’s claim is by giving the probability that he would make as few as 32 out of 50 free throws if he really makes 80% in the long run. The observed statistic is so unlikely if the actual parameter value is p = 0.80 that it gives convincing evidence that the player’s claim is not true.

6. Significance Tests: The Basics • The Reasoning of Significance Tests Based on the evidence, we might conclude the player’s claim is incorrect. In reality, there are two possible explanations for the fact that he made only 64% of his free throws. 1) The player’s claim is correct (p = 0.8), and by bad luck, a very unlikely outcome occurred. 2) The population proportion is actually less than 0.8, so the sample result is not an unlikely outcome. Basic Idea An outcome that would rarely happen if a claim were true is good evidence that the claim is not true.

7. Nature of hypothesis tests - How does a murder trial work? • First begin by supposing the “effect” is NOT present • Next, see if data provides evidence against the supposition Example: murder trial First, assume that the person is innocent. Then, you must have sufficient evidence to prove guilty Hmmmmm … Hypothesis tests use the same process!

8. PHANTOMS Steps: define the Parameter Hypothesis statements Assumptions/conditions Name the test Test statistic Obtain a p-value Make a decision State a conclusion in context

9. The “dull” hypothesis Significance Tests: The Basics • Stating Hypotheses A significance test starts with a careful statement of the claims we want to compare. The first claim is called the null hypothesis. Usually, the null hypothesis is a statement of “no difference.” The claim we hope or suspect to be true instead of the null hypothesis is called the alternative hypothesis. Definition: The claim tested by a statistical test is called the null hypothesis (H0). The test is designed to assess the strength of the evidence against the null hypothesis. Often the null hypothesis is a statement of “no difference.” The claim about the population that we are trying to find evidence for is the alternative hypothesis (Ha). In the free-throw shooter example, our hypotheses are H0 : p = 0.80 Ha : p < 0.80 where p is the long-run proportion of made free throws.

10. In any significance test, the null hypothesis has the form H0: parameter = hypothesized value The alternative hypothesis has one of the forms Ha: parameter < hypothesized value Ha: parameter > hypothesized value Ha: parameter ≠ hypothesized value To determine the correct form of Ha, read the problem carefully. Significance Tests: The Basics • Stating Hypotheses Definition: The alternative hypothesis is one-sided if it states that a parameter is larger than the null hypothesis value or if it states that the parameter is smaller than the null value. It is two-sided if it states that the parameter is different from the null hypothesis value (it could be either larger or smaller). • Hypotheses always refer to a population, not to a sample. Be sure to state H0and Hain terms of population parameters. • It is never correct to write a hypothesis about a sample statistic, such as

11. EXAMPLES: • Write the null and alternative hypotheses you would use to test each situation. • A governor is concerned about his “negatives” – the percent of state residents who disapprove of his performance. His political committee pays for a series of TV ads, hoping they can keep the negatives below 30%. They use a poll to test the ads’ effectiveness. • Is a coin fair? • Only about 20% of people who try to quit smoking succeed. Sellers of a smoking cessation program that their method can help people quit. p = .3 H0: HA: p < .3 H0: HA: p = .5 p ≠.5 p = .2 H0: HA: p > .2

12. Assumptions for z-test: Have a SRS Sample is less than 10% population np and nqare both at least 10 YAY! These are the same assumptions as confidence intervals!! Use p from the null hypothesis

13. Name the test: 1-proportion z - test

14. Test statistic (a.k.a. “z”): Use p from the null hypothesis

15. Significance Tests: The Basics • Interpreting P-Values The null hypothesis H0states the claim that we are seeking evidence against. The probability that measures the strength of the evidence against a null hypothesis is called a P-value. Definition: Assuming H0is true, it’s the probability that the test statistic (z) would take a value as extreme as or more extreme than the one actually observed is called the P-value of the test. The smaller the P-value, the stronger the evidence against H0provided by the data. • Small P-values are evidence against H0because they say that the observed result is unlikely to occur when H0is true. • Large P-values fail to give convincing evidence against H0because they say that the observed result is likely to occur by chance when H0is true.

16. If p is low, reject the Ho Statistically significant – • If p-value is small, “reject” the null hypothesis (Ho). • If p-value is large, “fail to reject” the null hypothesis (Ho).

17. Significance Tests: The Basics • How small does the p-value need to be in order for you to reject the null hypothesis? There is no rule for how small a P-value we should require in order to reject H0— it’s a matter of judgment and depends on the specific circumstances. But we can compare the P-value with a fixed value that we regard as decisive, called the significance level. We write it as α, the Greek letter alpha. • Usual values: 0.1, 0.05, 0.01 • Most common is 0.05

18. Definition: If the P-value is smaller than alpha, we say that the data are statistically significant at level α. Significance Tests: The Basics In that case, we reject the null hypothesis H0and conclude that there is convincing evidence in favor of the alternative hypothesis Ha.

19. Facts about p-values: • The p-value helps you make a decision about the null hypothesis • Large p-values show support forthe null hypothesis, but never proves that it is true! • Small p-values showsupport that the null is NOT true. • Doublethe calculator value for the p-value of two-tail(≠)tests • Never acceptthe null hypothesis! (“fail to reject”)

20. Never“accept” the null hypothesis!

21. At an  level of .05, would you ‘reject’ or ‘fail to reject’ H0 for the given p-values? • .03 • .15 • .45 • .023 Reject Fail to reject Fail to reject Reject

22. Draw & shade a curve & calculate the p-value: 1) right-tail test z = 1.6 2) left-tail test z = -2.4 3) two-tail test z = 1.8 P-value = .0548 P-value = .0082 P-value = (.0359)2 = .0718

23. When p-value is small Make a Decision: “Since the p-value (state p-value) is < a(state α) , Irejectthe H0.” Be sure to write Ha in context (words)! State a conclusion in context: “There issufficient evidence to suggest that Ha.”

24. When p-value is large Make a Decision: “Since the p-value (state p-value) is > a(state α) , Ifail to rejectthe H0.” Be sure to write Ha in context (words)! State a conclusion in context: “There is notsufficient evidence to suggest that Ha.”