Minnesota Mathematics Achievement Project (MNMAP)

1. Minnesota Mathematics Achievement Project (MNMAP) A Multi-institutional Study of the Relationship Between High School Mathematics Curricula and College Mathematics Achievement and Course-Taking Patterns Michael Harwell, Thomas Post, Amanuel Medhanie, Danielle Dupuis, Brandon LeBeau, & Debra Monson University of Minnesota SREE Meeting, March 4, 2010, Washington, DC

2. Overview of today’s comments: I. Background • Ongoing concern over the mathematics achievement of many U.S. high school students . • Role of high school mathematics curricula on college mathematics achievement and course-taking unclear. • Little evidence of the ability of different high school mathematics curricula to prepare an increasingly diverse student population for college mathematics , for a major in science, technology, engineering, or mathematics (STEM), or their relationship with students taking developmental mathematics courses in college.

3. II. High School Mathematics Curricula • Commercially Developed (CD) (traditional) Curricula • NSF-funded (Standards-based) Curricula (IMP, Core-Plus, MMOW). Significant criticism of NSF-funded high school mathematics curricula for college-bound students. • University of Chicago School Mathematics Project (UCSMP) Curricula

4. III. Literature Review • Schoen and Hirsch (2003) , Hill and Parker (2006), Harwell et al. (2008, 2009), Post et al. (in press) [all based on data from a single post-secondary institution]. • Mixed findings.

5. IV. Research Questions • (1) What is the relationship between high school mathematics curricula and, • student achievement (grade earned) in college calculus I, last college math course, likelihood that the difficulty of a student’s last mathematics course equaled/exceeded calculus I, and likelihood that the first college mathematics course a student completed was developmental? • can these relationships be generalized across post-secondary institutions varying in size, selectivity, and educational mission? • (2) Same as (1) using longitudinal college mathematics data.

6. V. Methodology Populations/Participants/Subjects: • The samples consist of approximately 13,300 students who completed at least three years of a NSF-funded , UCSMP, or CD curriculum, enrolled in one of 32 four-year post-secondary institutions in the upper Midwest of the U.S., and completed at least one college mathematics course. Research Design: • A retrospective cohort (quasi-experimental) cluster design was used in which archival data, including college mathematics data covering eight semesters, were obtained for students enrolled at one of 32 post-secondary institutions (clusters).

7. Student-level Covariates: • NSF-funded (1 = yes, 0 = no, CD = reference group) • UCSMP (1 = yes, 0 = no, CD = reference group) • Number of years of high school mathematics (3, 4, or 5) • ACT mathematics score • Sex (1 = female, 0 = male) • African American (1 = yes, 0 = no) • College major (Life Sciences/Health Sciences and Services 1 = yes, 0 = no; Humanities/Fine and Performing Arts/Social Sciences 1 = yes, 0 = no; Business 1 = yes, 0 = no; Other 1 = yes, 0 = no) with Engineering/Mathematics/Computer Science/Physical Sciences (STEM) serving as the reference group.)

8. Institutional-level Variables: • Size (enrollment = total number of full-time and part-time students enrolled) • Selectivity (ACT mathematics score of entering freshmen corresponding to the 25%ile) • Primary educational mission • full-time/more selective/high transfer-in rate • full-time/selective/high transfer-in rate • mixture of full-time and part-time students/mixture of selective and less selective/high transfer-in rate)

9. VI. Results From Table 1 (student frequencies, N=12,101) Educational Mission Select/ More Selective Select Less Select NSF 971 (55.8%) 754 (43.3) 16 (0.9) UCSMP 1292 (57.3) 950 (42.1) 12 (0.5) CD 4598 (56.7) 3486 (43.0) 22 (0.3)

10. From Table 5 Multilevel Cross-sectional (students within post-secondary institutions) Fixed Effects Results for College Calculus I Grades (K = 30, N = 3,649) Between-student Model EffectB Gender -.120* NSF-funded -.004 UCSMP -.020 Humanities -.217* Other -.324* * = statistically significant

11. From Table 6 Multilevel Results for Longitudinal Grade Data (grades within students within post-secondary institution) (K = 32, N = 10,288) Between-student Model: Intercepts EffectB African American - .28* Gender - .19* NSF-funded .02 UCSMP - .04 * = statistically significant

12. From Table 6 Multilevel Results for Longitudinal Grade Data (cont) (grades within students within post-secondary institutions) Between-student Model: Linear Slopes EffectB NSF-funded -.032 UCSMP .000 * = statistically significant

13. One other set of findings: • Simple percentages of students who completed a developmental mathematics course in college were 21.9%, 10.3, and 12.6% for the NSF-funded, UCSMP, and CD cohorts, respectively. • Multilevel modeling results showed that students in the NSF-funded cohort were almost twice as likely to take a developmental mathematics course initially in college relative to taking a more difficult course, compared to students in the CD cohort. • Multilevel modeling results showed that students in the CD cohort were about 1.3 times as likely to enroll in a developmental mathematics course initially relative to a more difficult course, compared to students in the UCSMP cohort.

14. VII. Conclusions • The results suggest that the NSF-funded, UCSMP, and CD curricula prepare students equally well for • college calculus • their last college mathematics course • longitudinal college mathematics course taking and success in these courses. • However students who completed a NSF-funded curriculum were more likely to take a college developmental math course. • These findings generally held for post-secondary institutions varying in size, selectivity, and educational mission.

15. VIII. Implications • For high schools, these results suggest that decisions about mathematics curricula for college-bound students should continue to be made based on factors believed to enhance mathematics learning and achievement such as available teaching expertise, state-mandated testing requirements, resources, etc., and not on rhetoric characterizing a curriculum as unable to adequately prepare students for college mathematics.

16. For post-secondary institutions, the findings suggest that the mathematics curriculum a student completes in high school is unlikely to predict success in mathematics courses (grades) or the pattern of course taking, but does predict whether a student’s first college mathematics course is developmental. But …

17. IX. Limitations • Fidelity of implementation of high school math curricula • Teaching quality and assessment • Missing data • Selection bias

18. X. What Now? • If the various curricula are preparing students similarly for college mathematics is this license to continue current instructional practices, which are largely defined by CD curricula with less than stellar student achievement and retention results over the past several decades? • Is there an argument for using NSF-funded curricula based on their links to NCTM-endorsed Standards that essentially document the “best practices” in the field?

19. Should the focus be on other facets related to how high school students perform in mathematics (e.g., motivation) and/or fidelity of implementation? • What is the role of the UCSMP curriculum in this discussion?

20. To receive a copy of this presentation or of the paper send an email request to mnmap@umn.edu Thank You!