Use, Misuse, and Missed Use of Quantitative Reasoning in Student Writing

1. Use, Misuse, and Missed Use of Quantitative Reasoning in Student Writing Nathan D. Grawe Carleton College With support from the US Department of Education’s Fund for the Improvement of Post-Secondary Education, the National Science Foundation, and the WM Keck Foundation.

2. What is QR?

3. What is QR? The habit of mind to consider the power and limitations of quantitative evidence in the evaluation, construction, and communication of arguments in public, professional, and personal life.

4. What is QR? Four facets of QR: 1) QR requires a basic skill set

5. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context "The test of numeracy, as of any literacy, is whether a person naturally uses appropriate skills in many different contexts" -National Council on Education and the Disciplines (2001)

6. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves argument

7. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves argument “Deploying numbers skillfully is as important to communication as deploying verbs.” -Max Frankel, The New York Times Magazine

8. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves argument “Numbers [are] the principal language of public argument.” -BBC Program More or Less

9. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves argument 4) QR is a habit of mind “[QR] is not a discipline but a way of thinking….” -Lynn Steen Achieving Quantitative Literacy

10. What is QR? “…sophisticated reasoning with elementary mathematics more than elementary reasoning with sophisticated mathematics.” -Lynn Steen Achieving Quantitative Literacy

11. What to the numbers show? “Consumption of green products that claim to be more environmentally benign has steadily increased in recent years.”

12. What do the numbers show?

13. How representative is that?

14. Compared to what?

15. How was the variable operationalized?

16. Is the outcome statistically significant?

17. What’s the effect size?

18. What’s the effect size?

19. What’s the effect size?

20. Are the results those of a single study or of a literature?

21. Are the results those of a single study or of a literature?

22. What’s the research design (correllational or experimental)? Fact: Those who work with computers earn 15-20% more than others. “Thus, computer training may, at least in the short run, be a profitable investment for public and private job training programs.”

23. What’s the research design (correllational or experimental)? Other interesting returns: Calculator = 12.8% Telephone = 11.4% Pencil/Pen = 11.2% Work while sitting = 10.1%

24. Controlling for what?

25. Carleton’s QuIRK Initiative • Situate QR in the context of argument • Look for evidence of QR proficiency in student papers

26. Our Population: The College Writing Portfolio Sophomore Writing Portfolio 3-5 papers plus reflective essay written in 2 of the 4 college divisions -observation -analysis -interpretation -documented sources -thesis-driven argument

27. Two Types of QR Use in Student Papers Central Use: Use of numbers to address a central question, issue, or theme Peripheral Use: Use of numbers to provide useful detail, enrich descriptions, present background, or establish frames of reference

28. Two Types of QR Use in Student Papers The importance of the periphery: “Even for works that are not inherently quantitative, one or two numeric facts can help convey the importance or context of your topic.” -Jane Miller The Chicago Guide to Writing About Numbers

29. Evidence from Student Writing Excerpt #: 1 Title: “Modeling the Implicit Learning of Language Acquisition” Thesis: Even if people aren’t aware of the systems undergirding our language, they implicitly learn the system through experience.

30. Evidence from Student Writing Excerpt #: 1 Discussion includes: • Critical analysis of effect size • Information on central tendency (mean and mode) • Information on extreme scores

31. Evidence from Student Writing Excerpt #: 2 Title: Denmark: A Modern Social Democracy” Thesis: Denmark is a model of social democracy for the rest of the European Community.

32. Evidence from Student Writing Excerpt #: 2 “world-class standard of living,” “high labor costs,” “high taxation,” “drastic changes” by right wingers in 1980s,“inflation brought under control,” “economy…fundamentally strong,” “since 1992…export performance slipped considerably”

33. Evidence from Student Writing Top 10 GDP per capita in $US, 2005 Excerpt #: 2 And Denmark reports the least income inequality of all countries measured by the UN.

34. Evidence from Student Writing Excerpt #: 3 Title: “Daycare and Development” Topic: Review of literature on the effects of day care on child development.

35. Examples from Student Work “With rising inflation…, many families are forced” to make tough child care choices. “Nonparental child-care is on the rise as more and more mothers choose to work….”

36. Examples from Student Work Alternative: “Understanding the effects of day care on child development is increasingly important as labor force participation among married women with children, ages 25-34, increased from 28% in 1960 to 68% in 2005. In fact, the number of working women with children nearly tripled during those 45 years from 8.1 million to 26.2 million.”

37. Evidence from Student Writing Excerpt #: 4 Title: “Democracy in India” Thesis: India represents a challenge to dominant political models of the determinants of democracy that can help refine that theory.

38. Evidence from Student Writing Excerpt #: 4 Uses of data: • Counts • Ratios • Percentages

39. The Need for Integrative Learning • 30% of gen ed papers are centrally QR relevant • Of these, nearly 50% are outside natural sciences • Another 25% are peripherally so • Of these, 90%+ are outside natural sciences and 50%+ are in Arts, Lit, & Humanities

40. The Need for Integrative Learning Rhetorical slant on QR engages faculty across curriculum QuIRK 18-month participation rate Overall: 65% Natural & Social Sciences: 75% Arts, Lit, & Humanities: 55%

41. The Need for Integrative Learning • Religion and American Public Life “…discussions across a variety of disciplines about the existing and proper place of religion in American public life (legal studies, journalism, sociology, history, public theology, and ethics)”

42. The Need for Integrative Learning • Classics 100 “…a close reading of an extended passage of Pausanias' Description of Greece. In it he records in great detail an inventory of all the statues of athletes (nearly 200) that he observed in the Sanctuary of Zeus at Olympia….”

43. The Need for Integrative Learning • Modern Indian Fiction 2 “…Salman Rushdie asserts the supremacy of Indian literature in English over all other language traditions…. [I will] anchor these discussions in a study of the actual numbers involved in publishing, readership, and language education. Students would be able to use the data both to contextualize claims such as Rushdie’s (just how many Indians read English? what fraction of Indian publishing is actually in English? how do writers in English fare in literary prizes awarded within India?) and to better illuminate an understanding of how complex the Indian literary landscape is.”

44. References • ABC News. February 21, 2007. “Do Libras Live Longer?” (http://abcnews.go.com/Technology/story?id=2890150&page=1) • Bialik, Carl. November 2006. “The Numbers Guy: Grading the Pollsters.” Wall Street Journal (http://www.mason-dixon.com/public/index.cfm?FuseAction=PressOffice.Article&ContentRecord_id=f1e66ca1-1372-fa49-99bd-58f6c5a4e505). • Brakke, David F. 2003. "Addressing Societal and Workforce Needs," in Quantitative Literacy: Why Numeracy Matters for Schools and Colleges, Bernard L. Madison and Lynn Arthur Steen, eds. Princeton, NJ: National Council on Education and the Disciplines. • De Lange, Jan. 2001. "Mathematics for Literacy" in Mathematics and Democracy: The Case for Quantitative Literacy, Lynn Arthur Steen, ed. Princeton, NJ: National Council on Education and the Disciplines. • Fishman, Charles. 2006. The Wal-Mart Effect. New York: The Penguin Press. • Frankel, Max. 1995. “Word and Image; Innumercy,” New York Times, March 5. • Hughes Hallett, Deborah J. 2001. "The Role of Mathematics Courses in the Development of Quantitative Literacy" in Mathematics and Democracy: The Case for Quantitative Literacy, Lynn Arthur Steen, ed. Princeton, NJ: National Council on Education and the Disciplines. • Miller, Jane E. 2004. The Chicago Guide to Writing about Numbers. Chicago: University of Chicago Press. • More or Less, British Broadcasting Corporation radio program. Retrieved April 27, 2007, from http://news.bbc.co.uk/2/hi/programmes/more_or_less/1628489.stm.

45. References (Cont.) • National Council on Education and the Disciplines. 2001. Mathematics and Democracy: The Case for Quantitative Literacy. Washington DC: Mathematical Association of America. • Newsweek. November 21, 2005. “How to Beat the Big Energy Chill.” (http://www.newsweek.com/id/51208) • Newsweek. August 6, 2007. “When Medical Studies Collide: Contradictory Reports? Meta-Analysis May Make Things More Confusing.” • Nurnberg, H. George; Paula L. Hensley; Julia R. Heiman; Harry A. Croft; Charles Debattista; and Susan Paine. 2008. “Sildenafil Treatment of Women with Antidepressant-Associated Sexual Dysfunction.” Journal of the American Medical Association, 300(4): 395-404. • (http://www.businessweek.com/magazine/content/07_32/b4045052.htm) • Reuters. November 26, 2007. “New Poll Shows Clinton Trails Top 2008 Republicans.” (http://www.reuters.com/article/politicsNews/idUSN2645320920071126) • Shafer, Jack. 2005. “Weasel Words Rip My Flesh! Spotting a Bogus Trend Story on Page One of Today’s New York Times,” Slate, September 20. • Steen, Lynn Arthur. 2004. Achieving Quantitative Literacy: An Urgent Challenge for Higher Education. Washington, DC: Mathematical Association of America. • Story, Louise. September 20, 2005. “Many Women at Elite Colleges Set Career Path the Motherhood.” New York Times. • Wallis, Claudia. 2007. “Is the Autism Epidemic a Myth,” Time, January 12. • WCCO. 2007. “Report: Indians Denied Home Loans More Often,” December 30. • Wiggins, Grant. 2001. "'Get Real!': Assessing for Quantitative Literacy" in Mathematics and Democracy: The Case for Quantitative Literacy, Lynn Arthur Steen, ed. Princeton, NJ: National Council on Education and the Disciplines.