Allan Rossman Beth Chance

1. Allan RossmanBeth Chance

2. Overview • What do want students to know and do at the end of the course • Our dream content • Top ten essentials • No client disciplines  • What would we cut to have time to get there • Assumptions about current content in many courses • Reality vs. fantasy • Are we there yet? • Example assessment items JSM 2010

3. #1 Understand the statistical process of investigation • Repeatedly experience the process as a whole 1. Formulate research question 2. Collect data 3. Examine the data 4. Draw inferences from the data 5. Communicate the results JSM 2010

4. #1 So what to cut? • Compartmentalizing the topics in the course • Data analysis, data collection, statistical inference • Instead: one categorical variable, compare two groups on quantitative response… • Some specific techniques • Example? Chi-square, ANOVA, regression • Possible out of class explorations JSM 2010

5. #2 Describe how to collect relevant data to answer research question • Research question vs. variable • Do the data answer the question • Example: Songs about the heart • “Worry questions” >> Make sure students have an opportunity to write their own research questions and to critique measurement/data collection methods JSM 2010

6. #2 So what to cut? • Ordinal, nominal, interval, ratio scales • Specifics of different sampling methods (cluster, stratified) and experimental designs • Though do make sure they realize not everything is an SRS or CRD • Acronyms! • Short-hand terminology (e.g., sampling distributions) and symbols (e.g., Ho/Ha) JSM 2010

7. #2 Assessment question • Pose a research question of interest to you that involves comparing two groups (but not one we discussed this quarter), • Identify observational units, explanatory and response variable(s), • Describe a detailed plan to collect data to investigate this question • Be sure to provide a detailed enough plan that someone else could carry out the actual data collection. • Explain whether (and why) your plan will involve random sampling and/or random assignment, or neither. JSM 2010

8. #3 Determine scope of conclusions based on data collection methods Random sampling: generalize to population Random assignment: cause/effect between explanatory and response variables Some studies use only one, some (few) use both, some (many) use neither >> Get students in habit of always commenting on both of these issues whenever they summarize the conclusions of a study. JSM 2010

9. #3 So what to cut? • Nothing; this point is too important • Move: Data collection issues to beginning of course, descriptive analysis of bivariate quantitative data to end of course • Students can discuss confounding variables in context of observational studies JSM 2010

10. #3 Assessment question • Students using cursive writing on the essay portion of the SAT in 2005-06 scored significantly higher, on average, than those who used printed block letters. • Can you conclude that cursive writing causes higher scores? Explain. • Different study: Identical essays were given to graders, some with cursive writing and some with printed block letters. Those with cursive writing scored significantly higher. • Can you conclude that cursive writing causes higher scores? Explain. JSM 2010

11. #4 Appreciate value/necessity of graphing data • Always start with a graph • Explain what see • Example: number of letters memorized • Make sure statements/conclusions about the data follow from the graph • Sometimes the graph is enough! JSM 2010

12. #4 So what to cut? • Pie charts • Choice of histogram bin width • But use technology explore different choices • Normal probability plots • Stemplots… • Boxplots!! JSM 2010

13. #4 Assessment question • Did distribution of inter-eruption times of Old Faithful change between 1978 and 2003? • If so, how? • How are changes favorable for tourists? • How are changes less favorable for tourists? • What other interesting features are apparent, have changed? JSM 2010

14. #5 Use proportional thinking • Especially important with categorical data, two-way tables • Conditional proportions • Proportion vs. percentage vs. percentage change vs. baseline risk vs. relative risk • Don’t need equal sample sizes to compare proportions or averages • Summary already takes sample size into account to produce a “fair” comparison JSM 2010

15. #5 So what to cut? • Formal probability rules, counting rules • Instead use two-way tables of counts, proportions • Bayes’ rule • Simpson’s paradox JSM 2010

16. #5 Assessment question • Data from murder trial of nurse Kristen Gilbert: • Of the 74 shifts with a death, 40 (54.1%) were Gilbert shifts, not significantly more than half. • Is this a reasonable calculation to perform here, to assess the evidence against Gilbert? Explain. If not, perform a more relevant calculation and explain why it’s more relevant. JSM 2010

17. #6 Develop distributional thinking • Conjecture how a variable will behave • Not everything follows a normal distribution • Example: Matching variables to graphs (ala ABS) • Appreciate the nature of variability • Think in terms of the distribution as an “aggregate” • Don’t let one value (data value or summary statistic) drive a conclusion • Focus on tendency, effects of outliers JSM 2010

18. #6 So what to cut? • Mode • Relative frequency distributions • Cumulative distributions • 1.5×IQR criterion for outliers • Details on calculating mean and median • Have to start making students responsible for having seen this before JSM 2010

19. #6 Assessment question Which would have more variability: ages of customers at McDonald’s near freeway or ages of customers at snack bar on campus? Explain. JSM 2010

20. #6 Assessment question Are pamphlets containing information for cancer patients written at an appropriate level that cancer patients can understand? Analyze these data to address the research question. Summarize and explain your conclusions. JSM 2010

21. #7 Consider variability in data when making comparisons • Comparing a particular outcome to a constant • Comparing outcomes in two different groups • Standardization can be a special case • Using a measure of variability to produce “ruler” for which we judge distances • Standard deviation (z-score) • Box lengths… JSM 2010

22. #7 So what to cut? • Calculation of standard deviation by hand • Short-cut calculation formulas (SD, correlation) • ANOVA table calculations • Linear transformations on summary statistics JSM 2010

23. #7 Assessment question Traffic Deaths year Sketch a graph of data from 1950-1960 where the change observed between 1955 and 1956 would be considered noteworthy. Now sketch a graph where the change observed between 1955 and 1956 would not be considered noteworthy. JSM 2010

24. #8 Consider variation of statistics when making comparisons • Averages vary less than individual values • Less and less with larger and larger samples • Larger samples give more precise estimates • Precision must be considered when making conclusions • Example: Three coin flips is not enough to decide whether a coin is fair JSM 2010

25. #8 So what to cut • Rules for means and variances • /n • Central Limit Theorem • Instead use simulations, graphs JSM 2010

26. #8 Assessment question In a rodeo roping contest, a contestant’s score is the average of two times. Explain why it is more fair to use this combination of two scores instead of relying only on one score. JSM 2010

27. #9 Understand the logic of inference • When can “chance” be eliminated as a plausible explanation? • Consider chance variability due to random sampling or random assignment • Strength of evidence vs. proof • Cobb (2007) argued that the reasoning process of statistical significance can best be introduced via simulation of randomization tests rather than normal-based models • “What if” distribution JSM 2010

28. #9 So what to cut? • Rejection region approaches • Tables of probability distributions • Randomization approach does not require probability distributions • Even with traditional tests, technology can calculate p-values, critical values • But still focus on well-labeled sketches of “what if” distributions • Technical conditions • 20-100% of specific (parametric) procedures JSM 2010

29. #9 Assessment question • MythBusters: Is yawning contagious? • Was MythBusters justified in concluding that the data provide strong evidence that yawning is contagious? • Conduct your own analysis • Explain reasoning process behind your conclusion JSM 2010

30. #10 Consider margin of error • Importance of interval estimate not only a point estimate • More than simply assessing statistical significance • Estimate + 2 SE • Focus on idea of interval of plausible values • Understand what parameter is being estimated • Issues that do/do not affect margin of error • Random sampling • Sample size • Population size JSM 2010

31. #10 So what to cut? • Solving algebraically for sample size • Any level other than 95% confidence • Any multiplier other than 2! • Interpretation of “confidence level” JSM 2010

32. #10 Assessment question • Suppose you want to estimate the proportion of the over 305,000,000 Americans who prefer cats to dogs within a 3% margin-of-error. Approximately what sample size would you need with a random sample? 10 1,000 100,000 1,000,000 10,000,000 JSM 2010

33. #1 Assessment question What type of study was this? Advantages and disadvantages? What graph could you examine to summarize these data? What is meant by “a 16 percent decreased risk of death”? What does it mean for the average life expectancy to be “significantly” longer? Is this an appropriate headline? Explain. JSM 2010

34. Conclusions • Fun to start from ground zero • What is your bare minimum of essential content? • Make sure “stat methods” courses don’t prevent “stat literacy” • Take advantage of computer/calculator power • Emphasize interpretation over calculation • Assess what you value JSM 2010

35. Questions? Allan Rossman arossman@calpoly.edu Beth Chance bchance@calpoly.edu http://www.rossmanchance.com/jsm2010.ppt JSM 2010