Generalizing From Samples

1. Generalizing From Samples Making Inferences

2. Generalizing From Samples • Sample error (Confidence intervals) • Confidence levels • Significance levels

3. Sample Errors • A.K.A. Confidence Intervals • Are a RANGE OF VALUES within which the population parameter most likely falls • Can be calculated from a given sample

4. Are expressed as: • Parameter = Statistic  ___% • Or, statistic (sampling error is ____ %)

5. Sample Error Example Table 1 Support For Tax Levy (%) For 45 Against 40 Undecided 15 1200 Total

6. Sample Error Example • 45% percent of the respondents indicated that, if the election were to be held today, they would vote for the levy (The poll has a 3% sampling error.) • If the election were to be held today, the “For the levy” side would carry 45% percent ( 3%) of the votes

7. Sample Error • Is affected by sample size (larger samples provide smaller sample errors -- see text, pgs. 187 & 188, Tables 7-1 & 7-2) • Can be calculated, if we are able to specify our desired confidence level

8. Confidence Level • A Statement of Likelihood (Probability) • In both of the previous cases, the statement means that the “true” percentage of levy supporters in the population is MOST LIKELY (PROBABLY) somewhere between 42% and 48% (Remember, sample had 45% “For”, with a 3% error rate)

9. Confidence Level • How Likely Is “Most (quite) Likely???” • The likelihood (probability) that the sample is representative of the population • The likelihood that we can safely infer from the sample to the population

10. Confidence Level • Is used with Univariate measures (percentages, typicality, dispersion) • Is affected by sample size (larger samples permit higher confidence) • Usually, we wish to be at least 95% confident • That our sample is representative • That the population parameter falls within the sample error

11. Confidence Level And Sample Error • Higher confidence means more sample error • Lower confidence means less sample error • For example, with (roughly) the same size sample, one might have • a 3% sample error and a 95% confidence level, or • a 4% sample error and a 99% confidence level

12. Confidence Level Example • If the election were to be held today, it is 95% likely that the “For the levy” side would carry 45% ( 3%) of the votes

13. Significance Level • A.K.A. Statistical Significance • Is used with bi- and multi- variate analysis (summary association measures, cross tabulations) • Is the likelihood that we are wrong (in error) if we conclude that the sample is representative of the population

14. Statistical Significance • Is not synonymous with importance (thinking that they are is a common error) • Is a tool used to help us decide whether or not it is likely that we are wrong if we decide that a sample is representative of the population

15. Significance Level: How It Works • There are many measures, etc. • All work by giving us the likelihood that we are wrong if we decide that the sample with which we are working is representative • All work by testing & trying to reject a null hypothesis (HO)

16. Hypotheses • Are testable (formal) statements • Which state the nature of the relationships between dependent & independent variables • When doing significance testing, we hypothesize re. the existence of a phenomenon in the population

17. Two Sorts Of Hypothesis • Test, or research, (a direct statement) HT • Directional (used when both variables are ordinal level or better) • Non-directional (used when at least one variable is nominal) • Null HO • Are the opposite of the test hypothesis • Are statements to the effect that there is no relationship between the independent and the dependent variables, or that sample findings are not representative of the population

18. Test Hypotheses • Education & income are related in the population (nondirectional) • Persons with more years of education tend to earn more money than do persons with fewer years of education, or education and income are positively associated in the population (directional) • Education and income are positively associated in the population, (directional)

19. A Null Hypothesis • There is no relationship between income and education in the population • This is what we test

20. Phenomenon Actually Does Exist In Population Y N

21. We Conclude That Phenomenon Exists In Population Y N

22. We Conclude That Phenomenon Exists In Population Y (Reject Null) N (Accept Null)

23. Phenomenon Actually Does Exist In Population We Conclude That Phenomenon Exists Y N In Population Y (Reject Null) N (Accept Null)

24. Phenomenon Actually Does Exist In Population We Conclude That Phenomenon Exists Y N In Population Y OK (Reject Null) OK N (Accept Null)

25. Phenomenon Actually Does Exist In Population We Conclude That Phenomenon Exists Y N In Population OK Y (Reject Null) N (Accept Null) OK In both cases, sample isrepresentative of the population

26. Phenomenon Actually Does Exist In Population We Conclude That Phenomenon Exists Y N In Population Type 1 Error Y (Reject Null) Type 2 Error N (Accept Null)

27. Phenomenon Actually Does Exist In Population We Conclude That Phenomenon Exists Y N In Population Type 1 Error Y (Reject Null) Type 2 Error N (Accept Null) Significance = The probability of making a Type 1 error (rejecting a correct null hypothesis)

28. Phenomenon Actually Does Exist In Population We Conclude That Phenomenon Exists Y N In Population Type 1 Error Y (Reject Null) Type 2 Error N (Accept Null) Significance = The probability of deciding that an unrepresentative sample is representative

29. Statistical Significance (Significance Level) • Usually, we want our findings to be significant at least at the .05 level (Closer to zero, such as .01 or .001, is better!) • One-tailed vs. two-tailed tests • Use a one-tailed test when direction is clear • When direction is not clear, use a two-tailed test

30. Reporting Significance • Include in text, or • Show at bottoms of tables, e.g.