Understanding Hypothesis Testing: A Comprehensive Guide

1. Mid-semester feedback • In-class exercise

2. Chapter 8Introduction to Hypothesis Testing

3. Logic of hypothesis testing • Use sample data to evaluate a hypothesis about a population parameter • Begin with known population and evaluate whether a sample that receives a treatment represents the known population or some other population • Did the treatment have an effect?

4. Logic of hypothesis testing • 4 steps in hypothesis testing • State hypotheses • Null hypothesis • Alternative hypothesis • Directional (one-tailed) • Nondirectional (two-tailed) • Set criteria for a decision • Probability that sample comes from population • Alpha level • Defines critical region(s) (region(s) of rejection) • Visualizing the boundaries for making a decision

5. Logic of hypothesis testing • 4 steps in hypothesis testing • State hypotheses • Set criteria for a decision • Collect data and compute sample stats • Compute M and convert to z-score • Make a decision • Reject the null hypothesis • Z-score in critical region • Probability of sample mean < alpha level • Fail to reject null hypothesis (retain null hyp) • Z-score not in critical region • Probability of sample mean > alpha level

6. Examples of hypothesis testing • I have taught statistics many times. Across all the students and all the tests the have taken, the mean score on my stat tests=80 with SD=10. Let’s assume that these represent known population parameters. • I decide to try something different in one of my stat classes. Twice a week, the students attend a tutoring session. I believe that the tutoring sessions will improve test scores.

7. Examples of hypothesis testing • 4 steps • State hypotheses • Set criteria for decision • Two-tailed test; =.05 • Collect data • M=85; n=25 • Make a decision (evaluate hypotheses) • Reporting results in the literature • Re-run analysis with a one-tailed test

8. Examples of hypothesis testing • In-class exercises • Are you guys really working harder (step 1) • Are you guys really working harder (step 2) • Are you guys really working harder (step 3) • Are you guys really working harder (step 4) • Reporting results in the literature • Re-run analysis with a one-tailed test

9. Uncertainty and errors in hypothesis testing • Inferences and sampling error • Type I error • Conclude an effect when there really isn’t one • Probability of Type I error = alpha level () • Type II error • Conclude no effect when there really is one • Probability of Type II error = beta ()

10. Assumptions for hypothesis test with z-scores • Random sampling • Independent observations • Data obtained from each individual not influenced by other individuals in sample • SD (variability) not changed by treatment • Normal sampling distribution

11. Effect size • Hypothesis testing indicates whether an effect is significant but does not indicate the absolute size of an effect • Large sample sizes can lead to statistical significance with small effects

12. Effect size • Cohen’s d is one measure of effect size • d=size of treatment effect in SD units • d=(M-)/ • Interpretation • 0 < d < 0.2 small effect • 0.2 < d < 0.8 medium effect • d > 0.8 large effect • Calculate effect size for tutoring study • =80; =10; M=85

13. Power • Power=probability of finding an effect assuming that one exists • Influenced by: • Size of effect • Alpha level • Sample size • Often used to determine appropriate sample size