t tests comparing two means

1. t tests comparing two means

2. Overall Purpose • A t-test is used to compare two average scores. • Sample data are used to answer a question about population means. • The population means and standard deviations are not known.

3. The Three Types • There are three ways to use a t-test in a comparative educational study: • One sample t-test • Independent t-test • Dependent t-test

4. Independent t-test • To compare two groups measured at the same time. • Treatment vs. Control at pre-test • Treatment vs. Control at post-test • Males vs. Females - ed. outcomes • Hi vs. Lo coping skills – stress outcomes

5. One sample t-test • To compare a sample to a population, or one group measured at one point in time. • Local Sample vs. national norm • Sample mean vs. some theoretically or practically meaningful cut point

6. Dependent t-test • To compare one group to itself over time, or one group measured at two times. • Pre-test vs. post-test for one group • The mean performance for the same group under two different drug dosage levels

7. Assumptions and Conditions • Normality • Homogeneity of Variance • Independence of Observations • Random Sampling • Population Variance Not Known

8. Testing Assumptions • Normality • Histograms and boxplots • Visual inspection • Reasonably within sampling error of normal • Homogeneity of Variance • SPSS gives you a test • p < .05 means assumption not met • If not met, you separate variance estimate t-test

9. Identifying Conditions • Independence of Observations • No score influences another score • Study design feature • Random Sampling • This is a formal assumption • It is also a study design feature • Population Variance Not Known • Sample data is used to make estimates of population parameters • Exception is the test value for the One Sample t-test

10. Additional Considerations • Confidence intervals can be used to test the same hypotheses. • There is a unique critical t value for each degrees of freedom condition.

11. Additional Considerations • Random assignment is needed to make causal inferences when using the independent t-test. • If intact groups are compared, examine differences on potential confounding variables.

12. Statistical Significance • How do you know when there is a statistically significant difference between the average scores you are comparing?

13. Statistical Significance • When the p value is less than alpha, usually set at .05. • What does a small p value mean?

14. Statistical Significance • You are most interested in whether the amount of difference you observe in your sample means is more than would be expected due to sampling error alone.

15. Additional Considerations • The z value of 1.96 serves as a rough guideline for evaluating a t value. • It means that the amount of difference in the means is approximately twice as large as expected due to sampling error alone. • Interpret the p value.

16. Statistical Significance • If the two population means are equal, your sample data can still show a difference due to sampling error. • The p value indicates the probability of results such as those obtained, or larger, given that the null hypothesis is true and only sampling error has lead to the observed difference.

17. Statistical Significance • You have to decide which is a more reasonable conclusion: • There is a real difference between the population means. Or • The observed difference is due to sampling error.

18. Statistical Significance • We call these conclusions: • Rejecting the null hypothesis. • Failing to reject the null hypothesis.

19. Statistical Significance • If the p value is small, less than alpha (typically set at .05), then we conclude that the observed sample difference is unlikely to be the result of sampling error.

20. Statistical Significance • If the p value is large, greater than or equal to alpha (typically set at .05), then we conclude that the difference you observed could have occurred by sampling error, even when the null hypothesis is true.

21. Hypotheses • Hypotheses for the Independent t-test Null Hypothesis: • m1 = m2 or m1 - m2 = 0 Directional Alternative Hypothesis: • m1 > m2 or m1 - m2 > 0 Non-directional Alternative Hypothesis: • m1 =/= m2 or m1 - m2 =/= 0 where: • m1 = population mean for group one • m2 = population mean for group two

22. Hypotheses • Hypotheses for the One Sample t-test Null Hypothesis: • m = m0 or m - m0 = 0 Directional Alternative Hypothesis: • m > m0 or m - m0 > 0 Non-directional Alternative Hypothesis: • m =/= m0 or m - m0 =/= 0 where: • m = population mean for group of interest (local population) • m0 = population mean for comparison (national norm)

23. Example • Our sample research design: • Head Start children randomly assigned to get in the program or be placed on the program’s waiting list. • Pre and Post test data collected on cognitive, social, and physical developmental outcomes • Measures scales with mean=50, SD=10.

24. Example • Our sample research design: • The subjects were not randomly sampled. They are volunteers. The parents of the study children applied for admission to Head Start in neighborhoods where more children apply than can be served. • There are no other publicly funded preschool programs in the area for low income children.