The Eyes Have It: Effective Data Visualization for Teaching Quantitative Literacy

1. The Eyes have it: Emphasizing Data Visualization When Teaching Students Meeting a Quantitative Literacy Requirement Robert Terry University of Oklahoma Teaching Statistics and Quantitative Methods Conference, Nashville, 2017

2. Background • Teaching students who: • need to meet a quantitative literacy requirement. • have a weak mathematics background. • will not likely take another statistics course. • are primarily in health-related majors.

3. Guidelines for Assessment and Instruction in Statistical Education (GAISE, 2005) • GAISE is based on the idea that a case-based course, replete with examples from the real- world, will do more to enhance your understanding of statistical thinking than a symbol-laden theory that one often gets in an introductory statistics course of this nature.

4. Six Recommendations • Following the GAISE Report (2005), we implement the following six recommendations for Introductory College Statistics. • Emphasize statistical literacy and develop statistical thinking. • Use real world examples whenever possible. • 3. Stress conceptual understanding rather than mere knowledge of procedures. • 4. Foster active learning. • 5. Use technology whenever possible. • 6. Use multiple assessments to improve and evaluate student learning.

5. Data Visualization Nicholas Lewin-Koh and Martin Theus, 2011.

6. Many Goals of Data Visualization • Computer Scientists want to: • Provide Insight into Data • Make Data More Accessible to Non-Professionals • Make Data Analytics Faster (Tableau Software) Robert Kosara, 2011.

7. Many Goals of Data Visualization • Statisticians want to: • Look for Patterns and Deviations from Patterns • Effectively Display Variability • Aim for Transparency • Stay Close to the Data Gelman and Unwin, 2011.

8. Many Goals of Data Visualization • Everyone wants to: • Use Visual Graphics to tell a story • Communicate Results Effectively • Use Graphics to Evaluate Claims • Use Visual Displays as Decision Aids

9. In Practice • Begin every class with a visual • Working in groups have students: • Construct a story • Create questions which arise from the graph • Evaluate a claim or make a decision

10. Beginning of Class

11. Has Violent Crime Increased In the Last 45 years? Steven Pinker Thinks Not! Pinker, 2011

12. Has Violent Crime Increased In the Last 45 years? • My Students Think So! (82%) • They are probably not alone Pinker ,

13. FIB Uniform Crime Reports 2016

14. 10.8% increase from 2015 Largest 1-yr increase since 1970-1971 (11.1%) FIB Uniform Crime Reports 2016

15. Student Discussion Points • Be careful of language – “largest increase in the US violent crime rate in 45 years” is not the same as “violent crime in the US has increased to its largest rate in the last 45 years” • Sensitivity to Change vs Absolute Level (Kahneman and Tversky) • When can we be sure a trend has turned for good? Kahneman and Tversky, 1968; L,opez, 2017; Butts, 2017.

16. Speaking of trends … • Has the climate really stopped warming since 1998? Glasslewis.com

17. Simple Linear Trend (until 2007) Fawcett and Jones,

18. Moving Averages (11-year) Fawcett and Jones 2008

19. Longer-term View Tamino, 2007

20. But what if … • You cherry-picked the data in such a way you started every “cooling” period at the end of a special heating cause (El Nino), and • Ended each cooling period before the next special heating cause started, and • Applied this strategy for the entire trend since 1973? • What would happen?

21. That’s right! Simpson’s Paradox and Climate Change https://thinkprogress.org/ten-charts-that-make-clear-the-planet-just-keeps-warming-ced872b4918c#.iftzemu77 Romm, Joe. Ten Charts that Make Clear the Planet Just Keeps Warming. ThinkProgress.Org. Jan 2013 [Retrieved Aug 2016]

22. Student Discussion • Moving averages seem to help eliminate the noise in the trend • Paying attention to “extreme points” can mislead • Does cherry-picking data always lead to a reversal paradox? • We just ended another El Nino Cycle .. .what happens next? • How, once again, can you tell if a trend has turned for good?

23. Taking a Deep Breath … cdc.Gov (2014)

24. Control Charts • Shewart (1931) and Deming (1943) • Special Cause Variation • Stable and Predictable Variation (Noise) • Common Cause Variation • Evidence of change in the system or our awareness of it (Signal) • It is not enough to do your best; you must know what to do, and then do your best. (Deming, 1964)

25. From Boggs, et al. The idea: Asthma as a process or system which is affected by multiple genetic and environmental events which can and should be identified from careful observation and experimentation. Like any such system, natural variation will exist and must be understood so as to maximize the efficiency of system intervention. Use of statistical quality control measures and graphs are useful in this regard. Failure to consider the normal history of the system using these methods can lead to poor outcomes. Boggs et al 1998

26. Here is the basic control chart (SD = sigma) Need to know: 1. mean PEFR 2. SD (Sigma) of the PEFR 3. Assume a normal distribution Lines on the chart conform to the 68-95-99.7 rule Mean +/- k*SD Where k is 1,2 or 3 units of SD

27. Example with filled in data from patient history Upper Normal process Limit (UNPL) is equal to : Mean + 3 SD Typical performance Lower Normal process Limit (LNPL) Mean - 3 SD Bad Day Good day Notice the natural variability in Respiratory Performance – this is not the case for Non-Asthmatics. VARIATION in performance is just as problematic for asthmatics as the LEVEL of performance

28. Rules for determining exposure to a Special Cause - Graphically

29. Decreasing natural variation with the addition of Fluticasone (Brand Name:Flonase) to normal care treatment Normal Medical Care Adding Flonase gives more stable respiratory functioning

30. Predicting Outcomes • Why do Students Leave at the End of their First-Year in College? • Sure, ACT and HSGPA are good predictors … but there has to be more …. • How might you use this information to predict retention outcomes? Pleitz, Terry, Fife, and Campbell, 2008; Terry, 2015.

31. 2006, 2007, & 2008 Cohorts N=7000+ Ss Archival: ACT, HS GPA, etc NSS: 100+ ITEMS, DURING SEP

32. Measurement Model

33. Logistic vs Linear Regression Model Terrry, 2915

34. Logistic Regression Results

35. Problems with Odds • Odds are hard to understand • Triple that for Odds Ratios (and Partial Odds Ratios) • But Odds Ratio keep showing up • Measure of risk for Retrospective Designs • Central parameters of Logistic Regression

36. ACT Scores and HSGPA as Risk Factors Note: Results shown controlling for High School GPA

37. HSGPA and ACTas Risk Factors Note: Results shown controlling for ACT Scores

38. Identifying the At-Risk Student HIGH RISK: ACT < 24 and High School GPA < 3.0 66.2 % Overall Retention Rate ACT avg of 22 Mean High School GPA of 2.94 6.21 % of Sample (N=504) High Risk: don’t meet academic reqs for admission

39. Identifying the At-Risk Student Low Risk: 90% total retention LOW RISK: Sliding Scale combination of ACT and High School GPA Scores 90% Overall Retention Rate Mean ACT Score of 28.80 Mean High School GPA of 3.92 26.50% of Sample (N=1860+)

40. Identifying the At-Risk Student Uncertain Risk UNCERTAIN RISK: 79.3% Overall Retention Rate ACT ave = 23.89 HS GPA = 3.54 67.29% of Sample

41. Identifying the At-Risk Student Low Risk: 90% total retention Uncertain Risk Probability of Retention High Risk: don’t meet academic reqs for admission Various predictors: ACT, HS GPA, etc.

42. Variables Used for Predictive Modeling • Admissions Variables • High School ACT • High School GPA • Class Rank • Demographic Variables • Gender • Home State • Alumni Ties • High School Class Size • Behavioral Variables • HS Academic Engagement • Institutional Commitment • Financial Concerns

43. Baseline Model Results shown controlling for High School GPA

44. Academic Engagement • While in high school how many times did you: • Feel overwhelmed by all you had to do? • Come to class late? • Come to class without doing your homework? • Feel bored in class? • Study outside of class?

45. Including Academic Engagementas an Additional Risk Factor Results shown controlling for High School GPA and ACT Scores

46. Adding Multiple Predictors • HS Academic Engagement • Financial Concerns • Alumni Ties • Institutional Committment

47. Including Multiple Predictors as a Additional Risk Factors Results shown controlling for High School GPA, ACT Scores, Financial Concerns, Class Size, Alumni Ties and Academic Engagement

48. Including a Statistically Insignificant Predictor (High School CLASS Rank) Results shown controlling for ACT Scores and High School GPA

49. Examples of Students with Extreme Changes in Percentages Results shown controlling for High School GPA, ACT Scores, Financial Concerns, Class Size, Alumni Ties and Academic Engagement

50. Examples of Students withExtreme Changes in Prob GPA: 3.06 ACT: 24 Commitment: 4.55 Financial: -3 Members: 0 Acad Engage: 3.24 Class size: 1250 Prob Change: 0.24 Results shown controlling for High School GPA, ACT Scores, Financial Concerns, Class Size, Alumni Ties and Academic Engagement