Adult ESL Literacy Impact Study: Description and Implications

1. Adult ESL Literacy Impact Study:Description and Implications Larry Condelli Stephanie Cronen American Institutes for Research, USA LESLLA Fifth Annual Symposium Banff, AB, Canada September 29, 2009

2. Overview of Presentation • Summary of Purpose, Research Questions and Design • Present descriptive data on students and teachers • Discussion and implications of potential findings

3. Evaluation of an ESL Literacy Intervention: Sam and Pat • Structured language approach • Adapted from Wilson Reading System • Never before evaluated for ESL • Literacy activities organized around basal reader/workbook • Heavily phonics-based

4. Sam and Pat: Instructional Approach • Direct instruction, transparent • Rules explained, modeled, practiced • Controlled text, vocabulary and grammar • Words match phonics already learned • Sequential • Easy to hard in defined steps • Multi-sensory, kinesthetic • ESL instruction to support literacy

5. Sam and Pat: Literacy and Language Skills Covered • Phonics for reading and writing • Sight words • Oral reading for accuracy and fluency • Reading comprehension • Vocabulary • Speaking and listening • Grammar

6. Part 1: Rereading familiar text Pre-reading (picture story) Letters and phonics Vocabulary and grammar Sight words Conversation Part 2: Review of Part 1 content Read story Written exercises Structured Two Part Lesson

7. Research Questions • How effective is instruction using the intervention in improving the English reading, speaking and listening skills of low-literate adult ESL learners? • Is the intervention more effective for certain groups of students (e.g., language, literacy level)? • Do differences in level of implementation of Sam and Pat and other instruction relate to variation in impacts?

8. Study Design • 10 adult ESOL centers across USA • Paired intervention and “normal” ESL literacy classes (34 total) • Random assignment of students and teachers • Minimum 5 hours/week 10-12 weeks instruction with approach • Other instruction also provided (5-10 hours/week) • Each class conducted twice over a year

9. Student Flow in the Study Intake NL Literacy • Student applies to center • Assessed for NLL • Recruited into study • Gives informed consent • Random assignment to class • Pretests administered • Instruction • Posttests administered Recruited into Study Informed consent Random Assignment Pre-test Post-test Instruction

10. Students Assessments • Phonics and decoding • Word attack • Letter/Word ID • Reading comprehension • Vocabulary (ROWPVT) • Listening, oral expression

11. Sam And Pat Teachers in the Study • All teachers randomly assigned • 3-day teacher training on curriculum • Follow-up visits by trainers • Classroom observations to monitor fidelity • Refresher webinar at start of second term

12. Literacy development instruction: Pre-literacy Phonics Fluency Reading strategies & comprehension Writing ESL Instruction: Oral language development Grammar, etc. Vocabulary Socio-cultural knowledge Functional literacy Classroom Observations

13. Students in the Study “True” Literacy (LESLLA) • Little or no literacy in native language • Limited oral English • Education: 0-6 years • Languages: • Haitian-Creole • Spanish • Burmese • Others

14. Students in the Study Non-Roman Alphabet Literate • Some Literacy in native language with non-Roman script • Mean education: 6 and more years • Limited oral English • Languages: • Armenian • Arabic • Farsi • Chinese • Others

15. Description of Sites • Ten sites participated: • 1 site in San Francisco CA area • 1 site in Los Angeles CA area • 3 sites in Miami FL area • 2 sites in Chicago IL area • 3 sites in Houston TX area • Sites included adult ed centers, high schools, and community centers • Type of programs included…??

16. Description of Students • 1,344 students participated for one term • They attended a range of low-level classes in the study, with ESOL Literacy being the most common: • ESOL Literacy • Low Beginning ESOL • Beginning ESOL (combined w/Literacy) • Students in the sample had been in the U.S. for an average of 4.7 years

17. Description of Students • Spanish was the most common language, followed by Armenian

18. Description of Students, Cont. • Average years of education • 6.7 years of education outside the U.S. • 7.7 years of education total • Why so high? • As a group, only Haitian students were “true literacy” • Spanish-speakers ranged from true literacy to beginning ESL level • Non-Roman alphabet students ranged from beginning ESL level to much higher

19. Description of Students, Cont. • Over half (58%) of Ss were female • Student ages ranged from 18 to 84

20. Teachers in the Study • 33 teachers participated in the study in Fall, and 31 participated in Winter/Spring • 1 class pair/2 teachers were dropped before the second term due to insufficient numbers of new students enrolling in their classes • 1 teacher taught 2 treatment classes both terms • Teachers had 7.5 years of experience teaching adult ESOL, on average • 5.3 years of experience teaching literacy-level

21. Teachers in the Study • The majority of teachers (85%) had some form of certification

22. Teachers in the Study • Most teachers’ highest degree received was either a Bachelor’s (42%) or a Master’s (48%) • 61% of teachers were male • Race/ethnicity varied • 40% white, nonHispanic • 30% Black or African American • 24% Hispanic or Latino • 6% Asian or American Indian

23. Teachers in the Study • Most teachers spoke a language other than English

24. Findings and Impacts….

25. SORRY!!! We’re not sure yet (and couldn’t tell you now anyway….)

26. Observations and Discussion Implications of Possible Findings for Practice and Research for LESLLA Students

27. Implications for LESLLA Practice • Let’s talk about the 3 most likely results scenarios: • Positive impact on decoding; no other impacts • No impacts on any outcomes • Negative impacts on some ESL outcomes

28. Implications for LESLLA Practice • Positive impact on decoding; no other impacts. Given Sam & Pat strong phonics focus this is the expected and most likely scenario. • But is improving only decoding after 3 months of instruction enough to warrant using it? • Would the impact differ by type of student? • What would the implications of these results be for teachers considering Sam & Pat or a similar approach?

29. Implications for LESLLA Practice • Gains for all but no differences between groups • Sam & Pat is no more effective than other types of ESL literacy instruction • Under what conditions, then, would Sam & Pat be a good choice for instruction? • What are some potential explanations for these results from your perspective?

30. Implications for LESLLA Practice • Negative impacts on some ESL outcomes • Some teachers expressed concern about taking 5 hours of instruction away from ESL activities, so we could see a negative impact on vocab or listening tests • How likely is this scenario? • What other explanations could we investigate in our analyses?

31. Context of LESLLA Research • The study took place in a challenging environment that makes having/finding an impact difficult, but that represents reality • Low exposure and instructional time in adult ESL class • Short class duration • Limited instructional time and irregular attendance • Training teachers • Lack of specialized training in literacy • Short training time available

32. Thank you! • Contacts • lcondelli@air.org • scronen@air.org • Enjoy Banff and LESLLA