280 likes | 393 Views
Section 10.3 Making Sense of Statistical Significance. February 15 th 2013. Choosing a level of significance (alpha level). How plausible is H 0 ? Depending on H 0 plausibility, you may choose a smaller alpha.
E N D
Section 10.3Making Sense of Statistical Significance February 15th 2013
Choosing a level of significance(alpha level) • How plausible is H0? • Depending on H0 plausibility, you may choose a smaller alpha. • If H0 is very plausible, you will need to have collect “more” evidence to reject it.
What are the consequences of rejecting H0? • If rejecting H0 would • costs lots of money • possibly cost lives • costs jobs • then alpha is usually very small
Fishing for significance • Let’s say were trying to find a connection between eating habits and intelligence. • Choose 40 foods, and assign people to increase the amount of the foods they eat, and see if there are any foods that make people smarter. • Of the 40 foods, we find that peeps and green beans make you smarter with alpha=.05. Is this a problem?
Section 10.4.1Inference as Decision February 15th 2013
When inference is used to make a decision… • Either you reject H0 or you fail to reject H0. • You can reject H0 correctly • You can fail to reject H0 correctly • You reject H0 incorrectly • (Type I error) • You can fail to reject H0 incorrectly • (Type II error)
Accept Ha Accept H0 H0 is true Ha is true
Potato Chips Example • The salt content of the chips should have a mean of 2 mg with a standard deviation of .1 mg. • When deciding whether to accept or reject a batch of potato chips, a company looks at the salt content of 50 chips. • If the salt content is too far away from the mean, it will reject the batch.
What range values are acceptable? • The company will check a 50 chip sample. • If our alpha is .05, the acceptable range is the same as the 95% confidence interval:
We understand with normal variation and everything working normally, we will get a sodium value between 1.9723 mg and 2.0277 mg 95% of the time. Accept or reject?
We understand with normal variation and everything working normally, we will get a sodium value between 1.9723 mg and 2.0277 mg 95% of the time. This means the 5% of the time you will reject a batch of chips that are fine. When we reject the batch (and H0) incorrectly we have committed a Type I error. Accept or reject?
Accept H0 Reject H0 Reject H0 95% Confidence Interval 1.9723 2.0277
Significance and Type I Error • The significance level α of any fixed level test is the probability of a Type I error. That is, α is the probability that the test will reject the null hypothesis H0 when H0 is in fact true.
Probability of the Type II error • What if there really is a difference in the overall saltiness of the potato chips, would always we see a “significant” result?
Accept H0 Reject H0 95% Confidence Interval 1.9723 2.05 2.0277
What is Power? • Power is a test of sensitivity. • Your statistical test may be able to detect differences, but how well does it detect difference of a pre-determined nature? • The Power procedure allows to state the probability of our procedure to catch the differences.
Power Procedure • Begin by stating your H0 and Ha as usual. • Find the z* or t* that would allow you to reject H0. • Find the x-bar that matches up with the z* or t*. • Assuming that you have a particular true mean, what is the probability that you would be to still reject the H0?
Power Example: Example 10.23 • Can a 6-hour study program increase your score on SAT? A team of researchers is planning as study to examine this question. Based on the result of a previous study, they are willing to assume that the change has σ=50. Research would like significance at the .05 level.
Power Example: Example 10.23 • A change of 50 points would be considered important, and the researchers would like to have a reasonable chance of detecting a change is this large or larger. Is 25 subjects a large enough sample for this project?
Step 1: State your hypothesis • H0: µ=0 • Ha: µ>0 • Where µ represents the change is in the SAT score.
Step 2: Find the z* value and find the data value • We'll set α=.05, invNorm(.95) gives us a z*=1.645. • What is the lowest x-bar would show significance? • Summary: If we had a study with n=25 and x-bar>16.45, we would have significance.
Step 3: Chance at importance • We stated that gains of 50 points would be considered "important". We state this as the alternative µ=50. • The power against the alternative µ=50 increase is the probability that H0 is rejected whenµ=50. • Restated: What the area from 16.45 to ∞ under a normal curve centered at µ=50.
Step 3 • normalcdf(16.45,1E99,50,50/√(25))=.9996 • Summary: because the power is so high, there is a great chance of finding a significance when the real increase is 50.
Increase Power by… • increase alpha • increase sample size
Exercises • 10.71-10.77 odd, 10.79-10.89