Education 793 Class Notes

1. Education 793 Class Notes Decisions, Error and Power 22 October 2003

2. Today’s agenda • Class and lab announcements • Questions? • Decisions, Error, Power

3. Review: Chain of reasoning Random selection can take several forms (such as simple, systematic, cluster, or stratified random sampling), and is intended to generate a sample that represents the population.

4. Four Steps • State the hypothesisH0 vs. HAlternate • Identify your criterion for rejecting H0 Directional or non-directional test One-tailed or two-tailed Set alpha level (Prob. incorrectly rejecting H0) • Compute test statistic  General form: Test = statistic – expected parameter standard error of statistic • Decide about H0

5. Million Dollar Question • To Reject or Not To Reject?

6. Review: Inferential error • Type I: Alpha • Rejecting the null hypothesis with the null hypothesis is really true • Type II: Beta • Failing to reject the null hypothesis when the null hypothesis is in fact false

7. Possible outcomes

8. Considerations • Basic considerations • What is the statistic of interest? • Is a one-tailed versus two-tailed test appropriate? • What is the nature of the sample? • Dependent versus independent • How much power is available? • Power is 1 – b • Factors influencing power: • Increasing the number of observations • Reducing the error in the data

9. Statistical Power • Power is the probability of correctly rejecting a false null hypothesis. As such, power is defined as: 1 - ß where ß is the Type II error probability • Two kinds of power analyses: • A priori: Used to identify what the sample size needs to be to identify a specified effect • Post hoc: Used when the null hypothesis has not been rejected, and when you want to know the probability that you have committed a Type II error

10. What Power is all about • Power analysis is about Type II errors, “missed effects” – failing to reject H0: when there really is an effect in the population • “Power” is the antithesis of “risk of Type II error” • Risk of Type II error = 1 - power • Power = 1 - Risk of Type II error

11. Trade off Between a and Power • Conundrum: As we decrease a (Type I Error), b (Type II Error) increases. The two errors have an inverse relationship. In order to deal with the problem, researchers follow an established set of guidelines. a=.05 or .01 • Which error is the most dangerous?

12. zcrit= -1.65 a=Type I Error Power of a Test • Example: Ho: m=72, H1: m<72 a=.05, n=36 Review: Given the sample data, a decision is made to reject or not reject the null hypothesis. This distribution is assumed to be true under the Null m=72

13. zcrit= -1.65 m=72 a=Type I Error Power of a Test • If the null hypothesis is false, then the above distribution is NOT the one we sampled from. We must have sampled from one of the possible alternative distributions, of which there are an infinite number. Recall H1: m<72

14. Power of a Test • The steps in computing power require the researcher to set a predetermined difference from the two means (D1) that is meaningful (Effect Size). • With three pieces of information, a, N, D, we can estimate the power of a test. • We will let computer programs do this for us.

15. Factors Affecting Power • Size of the difference between population means, what the book refers to as D1. • The greater the effect size (D1), the greater the power will be. Common sense, as the difference gets larger, it will be easier to detect. • Significance level • As the power of a statistical test increases so does a. This is the trade off between Type I and Type II Errors.

16. Factors Affecting Power • Variance • All other factors being equal (a, N, D), the smaller the standard deviation in the population, the greater the power. • Sample size • By increasing N, we decrease the standard deviation in the sampling distribution. Hence the power is increased.

17. Power as a function of… the true difference in  significancelevel samplesize

18. Next Week • Chapter 12 p. 333-367 and Available through JSTOR