1 / 21

Effectiveness of Reading and Math Software Products: Findings from First Student Cohort

This study explores the effectiveness of reading and math software products in volunteer schools. The key findings indicate that the use of these products did not significantly improve test scores, with few relationships found between effects and implementation factors.

johnnymorgan
Download Presentation

Effectiveness of Reading and Math Software Products: Findings from First Student Cohort

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Effectiveness of Reading and Math Software ProductsFindings From the First Student CohortMark DynarskiMay 2007

  2. Study Synopsis • Design • Nine reading and six math software products in 132 volunteer schools • Treatment teachers could use products, control teachers could not • Implementation • Companies provided training and other types of support • Study purchased various upgrades and hardware components • Key Findings • Test scores at the end of the school year were not statistically different • Few relationships between effects and implementation factors

  3. Study Size

  4. Implementation Framework • Student and teacher roles [O] • Teaching approach [O] • On-task behavior [O] • Consistency with suggested use of products [I] • Teacher training [O, R] • Amount of use [I, R] • Technical difficulties and teacher support [I] Key: O indicates observations, R records, I interviews

  5. General Implementation Findings • Nearly all teachers received training and believed it prepared them to use products • Difficulties using hardware mostly were minor • Total use of software products higher in treatment classrooms • When products are used • teachers more likely to be “facilitators” and students more likely to work on their own • more students on task in math classrooms

  6. Difference in Technology Use in Treatment and Control Classrooms: First Grade

  7. Effects on Classroom Practices Percent Difference: Teacher as Facilitator 100% 75% * * * 50% * 25% 0% First Grade Fourth Grade Sixth Grade Algebra Note: * Significantly different from zero at the 0.05 level

  8. Effects on Classroom Practices Percent Difference: Students On Task 30% 20% * 10% X 0% First Grade Fourth Grade Sixth Grade Algebra Note: * Significantly different from zero at the 0.05 level

  9. Estimating Effects • Outcome: spring test score • 3-level model (students, classrooms, schools) • 3-level model extended to estimate effects of conditions and practices (implementation) • interactions of treatment effect and classroom and school characteristics

  10. Effect Sizes By School: First Grade

  11. Test Scores: First Grade SAT-9 Reading Score 0.25 EffectSize 0.15 0.05 -0.05 OverallScore Soundsand Letters WordReading SentenceReading Note: * Significantly different from zero at the 0.05 level

  12. Test Scores: First Grade Test of Word Reading Efficiency Score 0.30 0.20 EffectSize 0.10 0.00 -0.10 OverallScore PhonemicDecoding Efficiency Sight WordEfficiency Note: None of the effect sizes is significantly different from zero at the 0.05 level

  13. Larger effects More experienced teachers Smaller student-teacher ratio No relationship Product usage Problems getting access Technical difficulties Computer specialist in school Professional development last year on using technology Poverty, urban area, African-American students, Hispanic, special education students Interactions: First Grade

  14. Test Scores: Fourth Grade SAT-10 Reading Score 0.30 0.20 EffectSize 0.10 X 0.00 -0.10 Overall Score Vocabulary Word Study Skills Comprehension Note: None of the effect sizes is significantly different from zero at the 0.05 level

  15. Larger effects Product usage No relationship Problems getting access Technical difficulties Computer specialist in school Professional development last year on using technology Poverty, urban area Interactions: Fourth Grade

  16. Test Scores: Sixth Grade SAT-10 Math Score 0.30 0.20 EffectSize 0.10 0.00 -0.10 Overall Score Problem Solving Procedures Note: None of the effect sizes is significantly different from zero at the 0.05 level

  17. No statistically significant relationships Interactions: Sixth Grade

  18. Test Scores: Algebra ETS Algebra Exam Overall Score Concepts Processes Skills 0.30 EffectSize 0.10 -0.10 -0.30 Note: None of the effect sizes is significantly different from zero at the 0.05 level

  19. Smaller effects when teachers had technical difficulties Interactions: Algebra

  20. Study Tradeoffs • 15 reading and math products studied • Many products and types of technology not in the study • Precision to detect small effect sizes • Average effect reported • Experimental design • Teachers have not used these products in current classrooms

  21. Second Year Study • 10 products, data from 77 schools, treatment and control teachers with new cohort of students • Will study relationship between teacher experience using products and effects • Effects reported for products

More Related