Assessing QL: Double Trouble

1. Assessing QL: Double Trouble Northeast Consortium for Quantitative Literacy Amherst College April 29, 2006

2. Plan for today • What I mean by assessment • What I mean by QL • QL forces in US society • Why US colleges & universities need to educate for QL • Challenges of education/assessment for QL • Changes needed • Can one play the game? • Assessment items

3. What is assessment? Comparing student learning with the learning goals of an academic program or curricular block of an academic program.

4. The Assessment Cycle • What should our students learn? • How well are they learning? • What should we change so that future students will learn more and understand it better? Answers three questions:

5. Why is assessment done? • The University administration mandated it. • The governing board mandated it. • The legislature mandated it. • The accrediting agency mandated it. • To evaluate academic programs. • To evaluate students. • To evaluate faculty performance.

6. What are better reasons for assessment? • To improve courses. • To improve academic programs. • To improve teaching. • To enhance student learning.

7. Kinds of Assessment • Classroom assessment • Formative assessment • Summative assessment • High stakes assessment • Outcomes-based assessment • Alternative assessment • Educative assessment • Assessment cycle • Authentic assessment

8. Assessment at different stages • Entrance stage – placement testing • Teaching and learning stage – formative feedback • End/exit stage – final grades, etc.

9. Assessment Cycle – Brief Version • Set learning goals and objectives. • Design strategies to accomplish objectives. • Determine areas and methods of assessment. • Gather assessment data. • Use the assessment data to improve program. • Do it again. Of course, there are epicycles when they are warranted.

10. Tests Surveys Evaluation reports Portfolios Essays Capstone courses Oral presentations Individual projects Focus groups Journals Senior theses Senior seminars Employer feedback Interviews Assessment Methods

11. References on Assessment • http://www.maa.org/saum (huge bibliography) • Assessment Practices in Undergraduate Mathematics, MAA Notes # 49, 1999. (Bonnie Gold, Sandra Keith, & Bill Marion, Editors) • Supporting Assessment in Undergraduate Mathematics, MAA Report, 2004. (Lynn Steen, Editor) • Assessment of Student learning in College Mathematics: Toward Improved Programs and Courses, AIR, 2006. (Bernard L. Madison, Editor)

12. Plan for today • What I mean by QL

13. What is QL/Numeracy? • QL to be the ability to understand and use quantitative measures and inferences that allow one to function as a responsible citizen, productive worker, and discerning consumer. (Madison - Notices of AMS) • QL is the ability to understand and use numbers and data in everyday life. (Madison – QL:Why Numeracy Matters) • QL is the ability to identify, understand, and use quantitative arguments in everyday contexts. QL describes a habit of mind rather than a set of topics or a list of skills. (Deborah Hughes Hallett) • QL is the capacity to deal with the quantitative aspects of life. (Lynn Steen)

14. Numeracy is the ability to handle numbers and data correctly and to evaluate statements regarding problems and situations that invite mental processing and estimating. (Jan de Lange) • QL relates to a cultural field where language and quantitative constructs merge and are no longer one or the other. (Bob Orrill) • Other (working) definitions of QL in: • Cockcroft Report (1982) • International Life Skills Survey (ILSS 2000) • Programme for International Student Assessment (PISA 2000). • QL is the knowledge and skills required to apply arithmetic operations, either alone or sequentially, using numbers embedded in printed materials; for example, balancing a checkbook, figuring out a tip, completing an order form, or determining the amount of interest from a loan advertisement. (US NALS) • “Numeracy” being replaced by “functional mathematics” in UK

15. Literacy has two very different meanings and this difference creates problems for QL education. • Literacy is often viewed as minimum competency as in “He is barely literate.” • Literacy is sometimes viewed as a high level of learning as in literati, or the educated class or intelligentsia.

16. Two Kinds of Literacy • Inert - Level of verbal and numerate skills required to comprehend instructions, perform routine procedures, and complete tasks in a routine manner. • Liberating - Command of both the enabling skills needed to search out information and power of mind necessary to critique it, reflect upon it, and apply it in making decisions. Lawrence A. Cremins, American Education: The Metropolitan Experience 1876-1980. New York: Harper & Row, 1988. (as quoted by R. Orrill in M&D)

17. Relatives of QL • Mathematical literacy • Numeracy • Symbol literacy • Document literacy • Financial literacy • Statistical literacy • Quantitative reasoning • Critical reasoning • Analytical reasoning • Scientific literacy • Symbol literacy • Technical literacy • Ethnomathematics • Functional mathematics

18. Kinds & Levels of QL • Critical QL – liberating QL • Functional QL – inert QL • Adult QL • Aging adult QL • QL for journalists • School level QL • College level QL • QL for social change • Environmental QL • Cultural/vocational QL – e.g. street vending, gambling, carpentry, real estate, & farming

19. Mathematics vs. QL Mathematics • Power in abstraction • Power in generality • Some context dependency • Society independent • Apolitical • Methods & algorithms • Well-defined problems • Approximation • Heavily disciplinary • Problem solutions • Few opportunities to practice • Predictable QL • Real, metamorphic contexts • Specific, particular • Heavy context dependency • Society dependent • Political • Ad hoc methods • Ill-defined problems • Estimation is critical • Interdisciplinary • Problem descriptions • Many practice opportunities • Unpredictable

20. QL Publications Why Numbers Count, (Lynn Steen, Editor) The College Board 1997 Mathematics and Democracy, (Lynn Steen, Editor), NCED 2001 QL:Why Numeracy Matters for Schools and Colleges, (Bernard Madison and Lynn Steen, Editors), NCED 2003 Achieving QL (Lynn Steen), MAA 2004 Mathematical Literacy by Eva Jablonka, Second International Handbook of Mathematics Education, Kluwer Academic Publishers, 2003, 77-104 Peer Review, Summer 2004 Alfred W. Crosby, The Measure of Reality: Quantification and Western Society, 1250-1600, Cambridge, 1997 Peter L. Bernstein, Against the Gods: The Remarkable Story of Risk, John Wiley & Sons, 1996 Pat Cohen, A Calculating People: The Spread of Numeracy in Early America, University of Chicago Press, 1982

21. Plan for today • QL forces in US society

22. QL Forces in US • Constitutional mandate for a census • Free market system -- minimally regulated labor markets • Democratic accountability • Representation of diverse interests -- minority representation • Informing political debate -- how much for how many • Confirming national identity -- what’s counted is what counts • De-regulation – more freedom to choose • Environmental protection and occupational safety • Emphasis on wealth accumulation • Heavy use of rapidly developing technology

23. Health care Retirement plans Rents and mortgages Investing Credit rates and plans Insurance Wages and salaries Education results Consumer protection Sports results Voting and polling Laws - regulation & de-regulation Court decisions Minority representations Taxes Federal & state budgets Economic policy Political debates Risk of terrorism Election procedures Computers have made data easy to gather and analyze. Data and numbers occur in many areas of daily life. Individual Decisions Decisions as Citizens

24. Plan for today • Why US colleges & universities need to educate for QL

25. High School College

26. High School and College Enrollments as a Percent of 14-17 &18-24 Age Cohorts (1900-1995) High School College

27. 20% of 4-year college seniors basic or below basic and only 34% scored as proficient on the NAAL in QL. AIR, 2006