The impact of writing letters to older peers

1. The impact of writing letters to older peers Shu-Ching Chang, Kun Chen04/21/2010

2. Research Purpose • To examine the impact of evidence-based letter-writing tasks on students’ conceptual understanding of some physics concepts. • In what way students benefit from the task of writing to different audiences.

3. Question • Do students who engage in writing letters to older peers perform better than students who do not?

4. Research Design • Two-year project. • SchoolClassStudents • Control and treatment groups. • Pre-/post-/delayed tests. • Tests: 20 multiple choice questions based upon 6 core concepts. ( Developed by Horizon Research Inc.)

5. Core Concepts

6. Writing Task—Three letters exchange between Grade 4 and Grade 11 Letter 1 Grade 11: Design a question to ask based on the concepts that will be needed by the 4th graders to complete the final design challenge. Grade 4: Make a hypothesis and/or claim to answer the question Letter 2 Grade 11: This letter will evaluate, critique the rationale, and suggest revisions to the elementary students’ ideas. Grade 4: Make a new hypothesis and/or claim to answer the question. Letter 3 Grade 11: Continue to evaluate, critique the rationale, and suggest revisions to the elementary students’ ideas. Grade 4: Same as response 2.

7. Guidelines for Grade 4 • Answer the question in the letter from your high school partners and explain your thinking by using supporting evidence. • Evidence can come from your experience and/or from the investigations you are conducting in science class. It can include the data you collect in experiments and your observations. • Consider different ways to explain your thinking about the concepts of force and motion. How could you use diagrams, pictures, charts or graphs to help explain your ideas? • If your high school partners suggest an experiment to test one of your ideas, think about how you can control the variable so that you have a fair test.

8. Data: participants • 4 elementary Schools in a north-eastern state of the U.S.A • 27 Teachers. Overlap in two years. Not randomly assigned. • Students are different from year to year. 4th graders.

9. Data: variables • Method (1: control 2: treatment) • School • Teacher • Student • Pretest Score • Posttest Score • Score type (0:pre 1:post 2:delayed) • Improvement: Post – Pre • Year (Combined) • Delayed test (Not available)

10. Data Analysis • ANCOVA (From our client) ---combing two years as a whole ---design structure is totally ignored. • Split-Plot Analysis --- can we combine the two years’ data? ---schoolteacher/classesstudents

11. ANCOVA Result F(1, 835)= 13.699, p<.000, effect size=0.25

12. Split Plot Analysis • Design schoolteacher/classesstudents • Combine two years’ data? --Students are different in two years. --Tests are standardized. -- Treatments are the same. -- Year effect is not significant.

13. SPA: Model 1 • Model1 :

14. SPA: Model 1 • SAS code: proc mixed data=data1; class School Teacher Student Method; model diff=School|Method ddfm = satterthoutp=diags; random Teacher(School Method) Student(Teacher); LSmeans School*Method / pdiffadj=Tukey; LSmeans School; run;

15. SPA: Model 1 • Result: • there is no significant difference of the score improvement for treatment and control groups in different schools.

16. SPA: Model 2 • Model2 :

17. SPA: Model 2 • SAS code: proc mixed data=data1; class School Teacher Student Method Year; model diff=Year|Method ddfm = satterthoutp=diags; random Teacher(Method) Student(Teacher); run;

18. SPA: Model 2 • Result: • there is no significant difference of the score improvement for treatment and control groups for two years.

19. SPA: Model 3 • Model3:

20. SPA: Model 3 • SAS code: proc mixed data=data2; class Teacher Method Student Scorecode ; model Score=Scorecode|Method ddfm = satterthoutp=diags; random Teacher(Method) Student(Teacher); LSmeansScorecode*Method / pdiffadj=Tukey; estimate "12-24" scorecode*method 1 -1 -1 1 run;

21. SPA: Model 3 • From the least square means, holding method fixed, the difference of pre and post tests is significant for control and treatment groups.

22. SPA: Model 3 • Results: • the interaction term of method and score-type turns out to be significant.

23. SPA: Model 3 • the improvement for students in the treatment group is significantly higher than the improvement for students from the control group. • This implies that new method seems to increase the critical thinking skills.

24. SPA: Model 4 • Model4(add school effect to model 3):

25. SPA: Model 4 • SAS code: proc mixed data=data2; class School Teacher Method Student Scorecode ; model Score=Scorecode|Method School/ ddfm = satterthoutp=diags; random Teacher(Method) Student(Teacher); LSmeansScorecode*Method / pdiffadj=Tukey; estimate "12-24" scorecode*method 1 -1 -1 1; run;

26. SPA: Model 4 • Results: • the school effect is not significant . • after adjusting for school effect, the interaction between method and score-type is significant.

27. SPA: Model 4 • the improvement for students in the treatment group is significantly higher than the improvement for students from the control group. • This implies that new method seems to increase the critical thinking skills.

28. Conclusion & Discussion • Students who engage in writing letters to older peers performed better than students who do not. • Teachers are not randomly assigned. • Delayed test.