1 / 36

Scott K. Baker, PhD Pacific Institutes for Research / University of Oregon Ben Clarke, PhD

Early Learning in Mathematics (ELM) The Efficacy of a Kindergarten Curriculum Implemented in Whole Classroom Settings. Scott K. Baker, PhD Pacific Institutes for Research / University of Oregon Ben Clarke, PhD Pacific Institutes for Research Hank Fien, PhD University of Oregon

spies
Download Presentation

Scott K. Baker, PhD Pacific Institutes for Research / University of Oregon Ben Clarke, PhD

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Early Learning in Mathematics (ELM)The Efficacy of a Kindergarten Curriculum Implemented in Whole Classroom Settings Scott K. Baker, PhD Pacific Institutes for Research / University of Oregon Ben Clarke, PhD Pacific Institutes for Research Hank Fien, PhD University of Oregon Keith Smolkowski, PhD Oregon Research Institute Chris Doabler, PhD Pacific Institutes for Research David Chard, PhD Southern Methodist University IES Conference June 2010

  2. Acknowledgements Institute of Education Sciences The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Goal 2 development grant, #R305K040081, and a Goal 3 efficacy grant, #R305A080114, to Pacific Institutes for Research. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. Additional Oregon Project Staff Kathy Jungjohann / Karen Davis: Curriculum development Mari Strand Cary / Rhonda Griffiths: Coordination and research Chris Doabler: Observation measurement and research

  3. Early Learning in Mathematics (ELM) 4-year randomized efficacy control trial Measuring the efficacy of a kindergarten mathematics curriculum in Oregon and Texas. Research Design Randomized Controlled Block Design Classrooms within school matched on full / half day and randomly assigned to treatment (ELM) or control conditions

  4. Purpose of the 4-year project • Study 1: Efficacy trial of the whole group curriculum (ELM) on kindergarten students’ mathematics achievement • Study 2: Efficacy trial of small group ELM component – Roots – on the achievement of at-risk students • Examine potential mediation variables, dose-response variables, and moderation factors

  5. Structure of the Curriculum • Daily Calendar Lessons / Activities • 15 minutes daily, whole class “circle” time • Monthly booklets with objectives and application activities • 120 Core Lessons divided into 4 quarters • 30 minutes whole class instruction • 15 minutes teacher directed written work • End of quarter assessment of progress

  6. ELM Instructional Content • National Math Advisory Panel (2008) recommends a focused, coherent progression of mathematics learning with emphasis on proficiency with key topics • ELM focuses on key strands rather than a broad array of mathematical content • Numbers and Operations • Geometry • Measurement • Vocabulary (NCTM Process Standard, 2000) NCTM Curriculum Focal Points for K (2006)

  7. ELM Conceptual Framework Development of Mathematical Concepts / Models Mathematics-related Vocabulary and Discourse Procedural Fluency

  8. Research questions • What is the immediate impact of ELM taught in general education kindergarten classrooms on mathematics achievement compared to standard district practice? • Is impact moderated by student level of risk for mathematics difficulties? • Does rate of teacher models or student practice opportunities mediate a condition effect? • Is there evidence of an interaction between condition and student practice on student outcomes?

  9. Study Sample • Assignment at the classroom level blocked on school • Districts = 3 • Schools = 24 • Intervention classrooms = 34 • Control classrooms = 30 • Students nested in classrooms • Whole class instruction • N = 1,349

  10. Student Demographics • 56.3% eligible for free or reduced lunch • 38.4% English Learners • 8.4% special education • 49.5% White • 36.4% Latino • 4.8% Asian / Pacific Islander • 2.3% African American

  11. Hypothesized Model of ELM Impact Intervention Mediators Proximal Outcomes Distal Outcomes

  12. Descriptive data on implementation • Classroom fidelity observations – treatment and control • Classroom observations focusing on instructional interactions – treatment and control • Teacher logs addressing content coverage – treatment and control classrooms

  13. Implementation Fidelity • General ratings (8 items) • Models skills/concepts appropriately and with ease • Engages students in learning throughout the lesson • Uses ELM / completes all lesson activities (dichotomous) • For each ELM activity (range 1-7 per lesson): Full (2) / Partial (1) / Not Taught (0)

  14. Implementation Fidelity Data: ELM Lesson Activities • 81 ELM (fidelity) observations during the year • Fall : mean = 1.71 (SD = .19) • Winter: mean = 1.65 (SD = .33) • Spring: mean = 1.62 (SD = .43) • Overall: mean = 1.65 (SD = .36) (83% of Full) • 2 of 81 lessons had a mean below Partial (1) level of implementation

  15. Student measures of impact • Test of Early Mathematics Ability (TEMA) • Early Numeracy – CBM • Oral Counting • Number Identification • Quantity Discrimination • Missing Number

  16. Method and Analysis Framework • Competing curricula • All students received instruction • Time balanced across conditions • Sample • At risk (some or high risk) • < 40th percentile • 66% of student sample • No risk • ≥ 40th percentile • 34% of student sample

  17. Nested Time × Condition Analysis • Outcome: net differences from pre to post • Nested students within classrooms • Control for nonindependence (e.g., ICCs) • Controls for teacher effects • Maximum likelihood (restricted) • Includes all cases with data at either T1 or T2 • Reduces bias from missing data • Moderation added Time × Risk × Condition interaction • Effect sizes: Hedges’ g

  18. Sample Means, SDs, and Ns

  19. TEMA Percentile Scores Gains by Condition • Gains • Control: 10.94 • ELM: 14.73 • Difference: 3.79 • Test of Condition • t = 2.10 • df = 61 • p = .0396 • ES = +0.14 • T1 differences were not statistically significant (t = 0.57)

  20. CBM Total Scores Gains by Condition • Gains • Control: 77.20 • ELM: 84.87 • Difference: 7.67 • Test of Condition • t = 1.99 • df = 61 • p = .0509 • ES = +0.14 • T1 differences were not statistically significant (t = 0.60)

  21. TEMA Raw ScoresCondition by Risk Status • Main Effects • Difference in gains: 1.32 • t = 2.41, df = 61, p = .0190 • Condition by Risk Status • t = 2.47, df = 61, p = .0162 • No Risk ≥ 40th %tile • Difference in gains: 0.04 • t = 0.51, df = 61, p = .9586 • Risk < 40th %tile • Difference in gains: 1.98 • t = 3.29, df = 61, p = .0017

  22. CBM Total ScoresCondition by Risk Status • Main Effects • Difference in gains: 7.67 • t = 1.99, df = 61, p = .0509 • Condition by Risk Status • t = 2.24, df = 61, p = .0289 • No Risk ≥ 40th %tile • Difference in gains: -0.27 • t = -0.05, df = 61, p = .9570 • Risk < 40th %tile • Difference in gains: 10.81 • t = 2.54, df = 61, p = .0138

  23. Effect Sizes (Hedges’ g) *p < .05; **p < .01

  24. Summary • ELM classrooms outperformed controls • TEMA raw and percentile scores • EN-CBM Total • Students at risk • Improve on all measures more than no-risk students • Control students at risk catching up on no-risk students • TEMA: 14.0 percentile gain on no-risk students • EN-CBM: 9.6 point gain on no-risk students • ELM students at risk catching no-risk students faster • TEMA: 18.6 percentile gain on no-risk students • EN-CBM: 20.63 point gain on no-risk students • No condition effects for students with no risk

  25. Preliminary Analysis of Association between Observation Data and Student Outcomes Intervention Mediators Proximal Outcomes Distal Outcomes

  26. Coding of Academic Teacher-Student Interactions (CATS) Observation Instrument • CATS uses a frequency count approach to measure teacher-student instructional interactions • Observers code behavior occurrences in a continual, serial fashion. • Utilizes a strict coding structure • CATS based on evidence of effective instruction in early literacy and beginning mathematics, and adapted from the STICO observation instrument (Smolkowski & Gunn, 2010)

  27. Hypothetical Case of a Instructional Interaction

  28. The Role of Teacher Modeling and Student Practice in Student Outcomes

  29. Preliminary Mediation Analysis • Does rate of teacher models or student practice opportunities mediate condition effect? • Rates of (a) teacher models, (b) student group practice opportunity and (c) individual student practice opportunities entered as mediators to determine if they decreased condition effect • Condition effect was still significant • No evidence to support this mediation hypothesis

  30. Secondary Analysis • If student practice overall is not mediating impact, perhaps the value (quality) of practice differs by classroom and is related to condition

  31. Interaction between Rate of Practice and Condition • By condition do rates of practice opportunities show the same pattern of impact on student outcomes? • Are treatment – control differences on student outcomes similar in classrooms with high rates of practice vs. low rates of practice?

  32. Interaction between Rate of Practice and Condition • Number of Classrooms by Treatment Condition and Median Rate of Individual and Group Practice Opportunities • Practice Opportunity Quartiles in Rate per Minute

  33. TEMA Scores by Rate of Practice • Within ELM condition • High-practice classrooms outperform low-practice classrooms • Difference = 2.59, t = 2.58, df = 29, p = 0.0151 • Within control condition • No difference between high- and low-practice classrooms • Difference = 0.39, t = 0.31, df = 29, p = .7575 • Within classrooms with an above-median practice rate • ELM classrooms (might) outperform control classrooms • Difference = 2.33, t = 1.85, df = 29, p = .0747 • Within classrooms with a below-median practice rate • No difference between ELM and control classrooms • Difference = 0.14, t = 0.14, df = 29, p = .8935

  34. EN-CBM Scores by Rate of Practice • Within ELM condition • No difference between high- and low-practice classrooms • Difference = 9.83, t = 1.46, df = 29, p = 0.1538 • Within control condition • No difference between high- and low-practice classrooms • Difference = -7.59, t = -0.92, df = 29, p = .3635 • Within classrooms with an above-median practice rate • ELM classrooms outperform control classrooms • Difference = 19.37, t = 2.35, df = 29, p = .0257 • Within classrooms with a below-median practice rate • No difference between ELM and control classrooms • Difference = 1.95, t = 0.29, df = 29, p = .7731

  35. Next steps • Have just completed implementation of Study 1 in Dallas, Texas • Have just completed implementation of Study 2 in Oregon • Will implement Study 2 in Dallas, Texas in 2010-11 • Ongoing analysis to investigate impact of condition and mediation and moderation variables associated with impact

More Related