Overview

2. Overview • Paper #1 - An efficacy trial of Word Generation: Results from the first year of a randomized trial • Joshua F Lawrence (University of California, Irvine), E. Juliana Paré-Blagoev (Strategic Education Research Partnership), Amy Crosson (LRDC, University of Pittsburgh), David Francis (University of Houston), Catherine E. Snow (Harvard University) • Paper #2 - Patterns of Students’ Vocabulary Improvement from One-time Instruction and Review Instruction • Wenliang He (University of California, Irvine), Emily Galloway, Claire White, Catherine Snow (Harvard University), Judy Hsu (University of Illinois at Urbana-Champaign) • Paper #3 Engaging Middle School Students in Classroom Discussion of Controversial Issues • Alex Lin (University of California, Irvine), Joshua F Lawrence (University of California, Irvine), Patrick Hurley (Strategic Education Research Partnership) • Paper #4 - Heterogeneous Treatment Effects for RedesignatedFluent English Proficient Middle School Students • Jin Kyoung Hwang (University of California, Irvine), Joshua F Lawrence (University of California, Irvine), Elaine Mo (University of the Pacific), Patrick Hurley (Strategic Education Research Partnership)

3. Academic Vocabulary Instruction Across the Content Areas: Results from a Randomized Trial of the Word Generation Program

4. Word Generation: Weekly Schedule Monday Paragraph introduces words Tuesday-Thursday Math-Science- Social Studies Friday Writing with focus words

5. Day 1 - Launch Introduction to weekly passage, containing academic vocabulary, built around a question that can support discussion and debate, (comprehension questions, student friendly definitions included)

6. Day 2 - Science Topic: not directly related to stem cell research but clearly a link could be made Target Words: investigate, theory, obtain Background Information: Countries have different views about citizens carrying guns. In some countries the import and export of guns is illegal. Subsequently, no citizen can obtain a gun in those countries (text continues). Questions: Are people more aggressive in countries that allow handguns? Hypothesis: Citizens of countries that allow handguns are more aggressive than citizens of countries that do not. Materials: Procedure: Data: Conclusion: What evidence do you have that supports your conclusion? Thinking experiments to promote discussion and scientific reasoning

7. Day 3 - Math 1. Some people believe that embryonic stem cell research is important. They think this because scientists use these cells to investigate diseases. Scientists try to find cures for these diseases, and for conditions like paralysis. Other people believe that embryonic stem cell research is wrong. They think this because scientists must destroy embryos to obtain these cells. In a recent poll, 40.75% of people said that the government should not pay for embryonic stem cell research. Which decimal is equivalent to 40.75%? A) 4.075 B) .4075 * C) .04075 D) .02 Mathematics problems using some of the target words: • Students can work in pairs • Whole class discussion • Open-response (show/explain how you got your answer)

8. Day 4- Social Studies • Positions: • Scientists should not be allowed to investigate cures for disease using stem cells from embryos. This is trying to “play God”. • Destroying an embryo to get the stem cells is murder. • The government should pay for embryonic stem cell research. This could lead to cures for many injuries and diseases. • Scientists should be allowed to do research on embryonic stem cells, but the government should not pay for it because many taxpayers oppose it. Developing positions on the issue set out in the passage, to help the class frame the debate. Note: these are optional. The class may want to develop its own positions!

9. Day 5 - ELA Writing Activity: Should the government pay for stem cell research? Give evidence to support your position.

11. Focus on classroom discussion • There are some studies on talk exposure and peer interactions and vocabulary for young children • Studies on discussion of older children have examined outcomes like reading comprehension, math, science, and philosophy content. • Connections to ELA and SS student outcomes is strong.

12. Research Questions • How engaging were classroom discussions in treatment and control schools? Were there differences in quality across content areas? Did schools that participated in the Word Generation program demonstrate improved classroom discussion? • Did participating in the Word Generation program impact students’ knowledge of the academic words taught? Did participation effect students’ general vocabulary knowledge? • Did improved classroom discussion mediate the impact of the Word Generation program on students’ academic vocabulary knowledge?

14. Support for Participation (1 – 3) • the teacher created a well-ordered and respectful environment that enabled engagement with lesson content and participation in the discussion • 1 was reserved for classrooms that were chaotic or there was student hostility or lack of participation • 3 points were scored if nearly all students appeared consistently engaged with minimal side talk and distractions

15. Student Engagement (1 – 3) • percent of students participating or attending to the classroom discussion. • 1 was awarded if around a quarter of the students participated in discussion during the observation period • 3 was awarded when 50% to 100% of students participated in discussion

16. Teacher Talk Moves (1 – 5) • Teachers’ use of open-ended questions and follow-up questions asking students to explain their thinking or provide evidence for their ideas. • 1 was given to classroom discussions in which all teacher questions had single, known answers (closed questions). • 5 reserved for classrooms where the teacher initiated a range of question types including open-ended questions and also asked students to provide evidence or explain their ideas more clearly.

17. Substantive contributions (1 – 5) • The level of students’ contributions to the discussion was rated on a five point scale. • 1 perfunctory answers were given the lowest ratings • 5 multiple students elaborated ideas and explaining their thinking while providing evidence. (In these classrooms students also asked each other to explain their thinking or explicitly link their own to others’ contributions.)

18. Composite Scores • Composite Discussion Quality Rating (1 – 4) – Overall quality score for each class. • Weighted School Level Discussion Quality Ratings (z score

20. Academic Vocabulary Knowledge • 36-item multiple-choice test

21. General Vocabulary Knowledge • Participants completed level 6 or level 7/9 of the Gates-MacGinitie vocabulary assessment (depending on their grade level)..

22. Covariates • Grade-level proficiency scores • School percent free and reduced lunch • School percent special education • Student grade level

23. RQ1 • How engaging were classroom discussions in treatment and control schools? Were there differences in quality across content areas? Did schools that participated in the Word Generation program demonstrate improved classroom discussion?

26. RQ2 • Did participating in the Word Generation program impact students’ knowledge of the academic words taught? Did participation affect students’ general vocabulary knowledge?

27. Treatment Effect • =

28. Effect Sizes (Student Level)

29. Basic Model • + + + +

30. HLM Predicting Student Knowledge of Academic Words

31. HLM Predicting Student knowledge of Gates Vocab

32. RQ3 • Did improved classroom discussion mediate the impact of the Word Generation program on students’ academic vocabulary knowledge?

38. Multilevel Mediation Model 0.396*

39. Multilevel Mediation Model 0.396* 1.633**

40. Multilevel Mediation Model 0.396* 0.533 1.633**

41. Multilevel Mediation Model 0.396* 0.533 1.633** indirect effect = .396*.533 = .211 total effect = indirect effect + direct effect = 0.211 + 1.28 = 1.48

42. Bootstrapping

43. Conclusions • Changes in curricular materials can improve discussion. • A free, low “implementation cost” program can have credible effects. • The proportion of total effect of Word Generation mediated through improved discussion is 0.14.

44. Limitation and questions • How can we further improve discussion? • Does improved discussion transfer? • What are the other effective pathways for student learning from WG?

46. Patterns of Students’ Vocabulary Improvement from One-time Instruction and Review Instruction Wenliang He, Joshua Lawrence (University of California, Irvine) Emily Galloway, Claire White, Catherine Snow (Harvard University) Judy Hsu (University of Illinois at Urbana-Champaign)

47. Background • Vocabulary knowledge is positively correlated with reading skills. • Prior research has focused on studies of factors that impact the learning of academic vocabulary (e.g. frequency, exposure, prior knowledge) and effective vocabulary intervention techniques (e.g. multimodal enhancement, active processing, mnemonics). • Regardless of the importance and the prevalence of vocabulary instruction in schools, few classroom-based intervention studies have been conducted on learning the effect of review (second-time) instruction.

48. Research Questions • What factors predict students’ improvement in vocabulary knowledge from first-time instruction in the WG program? • Can review (second-time instruction) significantly contribute to additional improvement in vocabulary knowledge to first-time instruction?

49. Research Design Group A 151 students Group B 159 students Pre-test Right before program started 5 words/week for 7 weeks 5 words/week for 7 weeks Review 7 Yellow Words 7 Red Words 5 words/week for 4 weeks 5 words/week for 4 weeks Post-test By the end of the 12th week

50. Research Design Overview of Yellow, Red and Core Instructional Words