Designing a Vocabulary Intervention with the Boston Public Schools

1. Designing a Vocabulary Intervention with the Boston Public Schools Strategic Education Research Partnership (SERP) American Educational Research Association Claire White April 10, 2007

2. Plan • Research Context: pilot schools • Assessment data: establishing the need • Survey data: teacher perspectives • Effective vocabulary instruction • “Word Generation” intervention • Pre-test results • Criteria for effective implementation of WG

3. Steps Forward Pilot WG Organizational Survey Now WG pre-test SERP sub-group develops WG GRADE data Teacher Survey Fall, 2005-2006

4. Westfield Middle School 80 % Black 16% Hispanic 1.8 White 1.6 Asian 29% Special Education MCAS Reilley Middle School 62% Black 18.1 % Hispanic 9.3% White 8.9 % Asian 25% Special Education MCAS Pilot Schools/Demographics

5. Massachusetts Comprehensive Assessment System (MCAS) Results 2006 (ELA) • Westfield • Reilley

6. GRADE data/6th grade • Reilley – 6th: 4.4 (mean stanine score) • 29% scored at the 3rd stanine or below (more than 1 standard deviation below the national mean) • Westfield- – 6th: 3.6 (mean stanine score) • 49% scored at the 3rd stanine or below (more than 1 standard deviation below the national mean)

7. Pilot Schools- Survey data • Reilley: high internal accountability; strong teacher responsibility for student learning, strong instructional leadership 2. Westfield: low internal coherence and accountability; weak teacher trust of colleagues and leadership; limited responsibility for student learning and student achievement

8. Leadership support for teachers

9. Leadership involvement with tracking academic progress

10. Teachers Share Beliefs About Mission

11. School profiles and assessment data confirm the need for an instructional intervention • The internal trust and cohesion level of individual schools provided us with context for intervention design and implementation (level of support, etc.)

12. Research on Vocabulary Shows… • Poor comprehension outcomes in middle school not necessarily a product of poor word reading but lack of vocabulary and academic language (e.g., Buly & Valencia, 2003; August & Shanahan, 2006) • Lack of knowledge of the middle and lower frequency “academic “words encountered in middle and secondary school texts impedes comprehension of those texts (e.g., Stahl & Nagy, 2006; Stanovich, 1986; Carlo, 2005)

13. Vocabulary Instruction in the Middle School Years • must be based on an understanding of the interrelatedness of content-area knowledge and academic language (e.g., Graves, 2006; Scarcella, 2003; Stahl & Nagy, 2006) • should occur through oral, reading, and writing activities throughout the content areas • should allow deeper explorations of language and should be rooted in text (e.g., Beck et al. 2002)

14. Challenges to Vocabulary Instruction Our initial classroom observations in BPS revealed: • Vocabulary is not usually taught • Instruction is fragmented between content areas • Texts fail to engage adolescents

15. Word GenerationProgram Goals: • Build the vocabulary of middle school students through repeated exposure to high frequency academic words in various contexts; • Promote regular use of effective instructional strategies among teachers; • Facilitate faculty collaboration on a school-wide effort.

16. Word GenerationProgram Features: • Focus on the Academic Word List (AWL); • Materials designed for flexible use across the curriculum; • An expectation that schools will dedicate at least 10 instructional minutes a day; • An opportunity for each school team to design a practical implementation plan that suits its own particular school context.

17. Word Generation: Materials • 20 weeks, each focused on a set of 5 words • Controversial topics include: global warming, censorship, dress codes and schools, steroids and sports, junk food and schools, the ethics of cloning, etc. Monday Paragraph introduces words Tuesday-Thursday Content-area word activities Friday Writing with focus words

18. Pre-launch Vocabulary Assessments 1. Vocabulary Self-Check (VSC) • student gauges his/her own level of knowledge about a word (40 items= 30 WG words and 10 non-words 2. Multiple Choice (Pre-WG) • 30 WG words chosen from 100 WG words to be taught over 20-week intervention “I do not “I have “I know something “I know it well know it” heard of it” about it” and can use it.”

19. Multiple Choice (Pre-WG) Sample items 1. She indicated that she was hungry. □ a. denied □ b. thought □ c. showed □. d. indeed 2. He will analyze the information. □ a. ignore □ b. anchor □ c. remember □ d. examine

20. Westfield MC Means and SD by Grade (n= 265)

21. Westfield: Multiple Choice Scores • Average Performance by Grade • 6th 60% correct • 7th 67% correct • 8th 72% correct • Total % Correct across Grades: 66%

22. Reilley MC Means and SD by Grade

23. Reilley: Multiple Choice Scores • Average Performance by Grade • 6th 63% correct • 7th 66% correct • 8th 66% correct • Total % Correct across Grades: 65%

24. Words Known by Fewer than 50% of Students

25. Self-report on the same words • interpret: up to 85% said they knew it well • sufficient: up to 74% said they knew it well • diverse: up to 86% said they knew it well

26. Where are we with WG? • Implementing Week 12 • Collecting and coding writing samples • Collecting and coding teacher feedback • Gauging effective implementation through various data sources

27. Criteria for Effective Implementation of WG • Focus on criteria for effective implementation and outcomes at: • Student level • Teacher level • School level

28. Student Level • Effective use of words in natural interactions and weekly writing paragraph • Improvement in content-area language and vocabulary • Improved performance on post assessments

29. Teacher Level • Improved knowledge of effective vocabulary strategies • Increased responsibility for teaching content through language and language through content • More opportunities provided by teachers for students to use academic language • Productive feedback/sharing of concerns through meetings and weekly evaluations on WG materials and activities for revisions

30. School Level • Higher level of cohesion and internal accountability (cooperation across the content areas) • Greater involvement by principals in disseminating and overseeing intervention • Shared commitment by leadership, teachers and students to developing and sustaining a school-wide word culture

31. Next steps • Continued documentation of effective implementation (classroom observations, interviews, video-recording) and effective instructional practices and word learning • Post –survey? • Post-assessment(May/June 2007)

32. Thanks to • Joanna Christodoulu- Harvard Graduate School of Education • Michael Kieffer- Harvard Graduate School of Education • Michelle Forman- Harvard Graduate School of Education • Jeannette Mancilla-Martinez- Harvard Graduate School of Education • Sarah Meacham- SERP • Lasse Isakson- Harvard Graduate School of Education • Catherine Snow- Harvard Graduate School of Education • Jennifer Zeuli- Harvard Graduate School of Education