Intro to Stats

1. Intro to Stats Significance Testing

2. Hypothesis Test • A statistical method that uses sample data to evaluate a hypothesis about a population • 1. State a hypothesis • 2. Use the hypothesis to predict the characteristics the sample should have • 3. Obtain a random sample from the population • 4. Compare the obtained sample data with the prediction made from the hypothesis

3. “Significant” • Suggests that the difference or relationship is systematic and not due to chance • A significance level gives the risk that the effect is due to chance • At p < .05, there is about a 5% (5/100) chance that an effect just happened in a normal distribution and is not related to any other variable

4. p < .05 for 2-tailed occurs at z = 1.96

5. Qualifying factors • Difference in size between the sample mean and the original population mean • The variability of scores • The number of scores in the sample

6. Assumptions Made • We make certain assumptions about the impact of the methods used to obtain the data sample • Random sampling • That the sample represents the population • Independent observations • The first measurement has no relationship to the probability of the second measurement • The value of variability is unchanged by the treatment • In most hypothesis-testing situations, we do not have the “original” population variability • A normal distribution

7. Never Sure • Researchers can never be sure that their hypothesis is “true” • Sample may not perfectly reflect the population • Other influences (confounds) may cause the results • It just might be one of those few chances • These concerns are lessened every time that a finding is replicated

8. Can lead to errors • The null is rejected, but there really is no effect (Type I error) • Or fail to reject the null (accept null), but there really is an effect (Type II error)

9. Never Sure 2 • Never know the truth about the null hypothesis • The likelihood of a Type I error is defined by the level of significance • p < .05 means there is a 5% chance of rejecting the null when the null is true (conclude there is a difference when there is none) • Type II error is related to power and sample size

10. Effect sizes • Rather than rely on hypothesis testing, it is now typical to include effect sizes • Most hypothesis tests simply state that a finding is UNLIKELY (or not) • Significance does not mean it’s important • Effect sizes give a way to capture the SIZE of an effect and make stronger statements about the relationship • This also sidesteps issues of power • Power is strongly linked to sample size and therefore effects may be underestimated in small samples and overestimated in large samples

11. Significance vs. Meaning • Studies may show significant differences but the differences may not be meaningful • Small differences in large samples • Small differences that come with a large cost • Studies may not show significance but the differences may be meaningful • Small differences in small samples • Small differences that come with a big benefit and little cost

12. Inferential Statistics • Inferences made about the population based on a sample • Which test determined by • Continuous vs. categorical variables • Number of variables • Whether variables vary between subjects or within subjects *Your book has a useful chart

13. Setting the Stage • 1. State the null and research hypotheses • 2. Set the level of risk (usually .05) • 3. Select the appropriate test statistic • 4. Compute the test statistic • 5. Determine the critical value for rejection of the null • 6. Determine whether the statistic exceeds the critical value (usually at p < .05) • 7&8. If over the critical value, the null hypothesis is unlikely THEREFORE effect must be due to other variable • If not over the critical value, the null is accepted • INTERPRET

14. One-sample z-tests • One-sample z-tests are inferential statistics that allow you to compare a mean from a sample to the average of a population.

15. Example 1 • 1. State hypotheses • Null hypothesis: there is no difference between civic engagement in Texas A&M students and the national average • H0: XA&M = μstudents • Research hypothesis: there is a difference between civic engagement at A&M and the national average • H1: XA&M ≠ μstudents

16. Calculating Z = X – μ SEM X = mean of sample μ = population average SEM = standard error of the mean SEM = σ/square root of n σ = standard deviation for the population n = size of the sample

17. Example 1 • Civic engagement of A&M students • X = 3.42 organizations, n = 200 • μ = 1.5 organizations, σ = 1.2

18. Example 1 • Civic engagement of A&M students • X = 3.42 organizations, n = 200 • μ = 1.5 organizations, σ = 1.2 • SEM = σ/square root of n

19. Example 1 • 5. Determine the critical value for rejection of the null • We know that a z of 1.96 or larger is significant at p < .05 • Remember this is the smallest value of z that corresponds to a p of .05 or less (falls in the “extreme range” • Bigger zs = more likely to be a significant difference

20. Example 1 • 6. Determine whether the statistic exceeds the critical value • 24 > 1.96 • So it does exceed the critical value • THE NULL IS REJECTED IF OUR STATISTIC IS BIGGER THAN THE CRITICAL VALUE – THAT MEANS THE DIFFERENCE IS SIGNIFICANT AT p < .05!! • 7. If not over the critical value, fail to reject the null

21. Example 1 • In results • Student participants at Texas A&M University joined more civic organizations per year (M = 3.42) than the national average, z = 24.00, p < .05.