Building a Knowledge Base for Teaching Adult Decoding

1. Building a Knowledge Base for Teaching Adult Decoding Richard Venezky, Principal Investigator Deborah Knight, Co-Investigator University of Delaware Judith A. Alamprese, Co-Principal Investigator Fumiyo Tao, Field Study Director Abt Associates Inc. Bethesda, MD

2. Study’s Research Questions • How does the use of enriched decoding strategies affect adults’ development of decoding skills? • What is the relationship between adults’ development of decoding skills and their development of fluency and comprehension? • How are adults’ background characteristics and their amount of participation in reading instruction related to the improvement of their reading skills? • How is the type of reading instruction that adults receive related to their improvement of reading skills?

3. Overview of Study • Phases of study activities • Intervention conceptualization (Year 1) • Design Studies (Years 1-2) • Field Study (Years 2-4) • Analysis and report preparation (Year 5) • Experimental design • Naturalistic study in operating adult basic education programs • Approach of “use-inspired basic research” (Stokes, 1997) • Target population: adults with 4-7th grade-level equivalent in comprehension

4. Dependent Mediating Variables: Independent Variables: Independent Variables: Variables: Learner Learner Characteristics Programmatic Characteristics: Overall Program Programmatic Characteristics: Overall Program Outcomes Operations and Instruction Operations and Instruction Overall Program Operations Overall Program Operations Learner’s Background Learner’s Background Short Short - - Term Term (3 (3 - - 12 months) 12 months) • • Demographic Demographic characteristics characteristics • • Changes in Changes in Program Program Program Program Program Program Support Support Support learning learning Management Management Management Recruitment Recruitment Recruitment Services Services Services • • Educational and Educational and and Intake and Intake and Intake training experience training experience • • Personal growth Personal growth • • Work experience Work experience • • Health background Health background Program Program Program Assessment Assessment Assessment Program Program Program Improvement Improvement Improvement Articulation Articulation Articulation Learner Persistence Learner Persistence • • Total participation hours Total participation hours • • Duration of participation Duration of participation Instruction Instruction Instruction Learner Development Learner Development • • Self empowerment Self empowerment enhancement enhancement Framework for Study Design Dependent Dependent Mediating Variables: Mediating Variables: Independent Variables: Independent Variables: Variables: Learner Variables: Learner Learner Characteristics Learner Characteristics Programmatic Characteristics: Overall Program Programmatic Characteristics: Overall Program Outcomes Outcomes Operations and Instruction Operations and Instruction Overall Program Operations Overall Program Operations Learner’s Background Learner’s Background Short Short - - Term Term (3 (6 - - 12 months) 12 months) • • Demographic Demographic characteristics characteristics • • Changes in Changes in Program Program Program Program Support Support reading skills learning Management Management Recruitment Recruitment Services Services • • Educational and Educational and and Intake and Intake training experience training experience • • Personal growth Personal growth • • Work experience Work experience • • Health background Health background Program Program Assessment Assessment Learner Program Improvement Improvement Articulation Articulation Learner Persistence Learner Persistence • • Total participation hours Total participation hours • • Duration of participation Duration of participation Instruction Instruction Learner Development Learner Development • • Self empowerment Self empowerment enhancement enhancement 3

5. Identify Decoding Interventions Conduct Design Studies Conduct Field Study • Review K-12 and adult reading literature • Examine results from Abt Associates’ reading study • Generate hypotheses about role of decoding in learning to read and test in situ; vary aspects of instruction • Conduct 4 cycles of testing interventions in 4 classes • Collect data over 2 program years • Assess enriched decoding instruction compared to existing approaches and to comparison group of structured reading classes • Sample of 21 ABE programs, 30 reading classes, 600 learners • Randomly assign 7 programs to decoding intervention and 7 to control group • Train treatment instructors in decoding intervention • Data collection: • 2 years for treatment group • 2 years for control group • 3 years for comparison group • Identify types of decoding methods that might be tested in Design Studies • Refine reading intervention based on results of each cycle Study Design and Phases 4

6. Design Studies Purpose • Generate hypotheses about role of decoding in adults’ learning to read and test them in situ, so that practices can be implemented in actual instructional contexts • Systematically vary aspects of adult decoding instruction to determine the most effective practices for the field study

7. Design Studies Questions Guiding Design Studies • How do rule-based (deductive/abstract) and exemplar-based (inductive/analogical) instructional methods affect adult learners’ development of decoding skills? • How much and which metalinguistic knowledge enhances adult learning? • Does reading fluency increase differentially if adult learners participate in repeated readings that emphasize speed versus accuracy? • Do adult learners’ decoding skills improve when they are given spelling practice (investigation in progress)?

8. Design Studies Sample and Methodology • Sample • Four ABE classes • Three ABE instructors, 2 university researchers • 26 adult learners • Gender: 17 females, 9 males • Race/Ethnicity: 13 African Americans, 9 white non-Hispanic, 1 white Hispanic, 2 Asian, 1 mixed race • Age: Mean=25 years • Methodology • 4 cycles of intervention studies focused on specific questions during each cycle • Continued refinement of instruction through each cycle based on results of previous cycle and new question

9. Design Studies Features of Interventions Examined in Design Studies • Learner outcomes (determined by embedded assessments, observations) • Classroom practicality • Learner engagement • Instructor responsiveness and effectiveness

10. Design Studies Findings from Design Studies To Date • Learners benefit from a combination of rule-based and exemplar approaches • Learners are interested in and can apply metalinguistic information about orthographic patterns and etymology, but this information needs to be presented clearly with a minimum of complexity • When asked to read for speed and accuracy, adult learners sacrifice accuracy for speed if they are timing each other using a stopwatch • Adult learners are motivated to learn to spell, and often believe that they can read words better than they actually do • Instructors will use scripted reading lessons

11. Intervention Curriculum Based on findings from Design Studies, the intervention curriculum will: • Consist of 40 lessons, including 10 review; each lesson will be 35 minutes (integrated into a reading class) and include a mixture of reading and spelling patterns • Emphasize metalinguistic knowledge that is of high utility or that explains why English behaves the way it does • Include spelling tips • Include a component designed to improve accuracy and speed of reading • Embed assessment in the lessons (except review) for monitoring learner progress

12. Intervention Curriculum The intervention curriculum will: • Include periodic optional practice lessons that do not introduce new concepts so that differentiated instruction can be offered • Include an instructor’s guide with scripted lessons

13. Intervention Curriculum Format of Lessons • Review of prior learning • Use of an authentic document • Introduction of new material in a mini-lesson • Use of guided practice reading and spelling words • Exercises in fluency/accuracy and/or spelling • Independent practice/embedded assessment

14. Field Study Purpose:Conduct an experiment in situ of the effectiveness of the intervention curriculum in improving adult learners’ reading skills Sample • 21 ABE programs, approximately 30 ABE reading classes and 27 instructors; approximately 600 adult learners • Program meets a basic level of operational planning, organization, and structure 13

15. Field Study Sample • 7 comparison ABE programs that use commercially produced, structured reading programs • 7 ABE programs randomly assigned to treatment group (enriched decoding); instructors will be trained to use intervention curriculum • 7 ABE programs randomly assigned to control group; instructors will conduct reading classes as they usually do

16. Field Study Measures: Adult Learner • Decoding, spelling, vocabulary, and comprehension tests: WRAT-3 (Word Reading and Word Spelling), Woodcock-Johnson (Word Attack, Letter-Word Identification), Nelson Reading Comprehension and Word Meaning Tests • Reading fluency measures: TOWRE (Sight Word Efficiency, Phonemic Reading Efficiency) • NAAL Passage Reading • Background Interview: demographic characteristics, education and training, work experience, health status • Self-empowerment scales: Abt’s Adaptation of the General Self-Efficacy Scale and Rosenberg’s Self-Esteem Scale

17. Field Study Measures: Adult Learner • Attendance: hours and weeks per program year; participation in other ABE classes during same period Measures: Instructor • Reading Instructional Content, Strategies, and Materials: Abt Associates’ Class Observation Instrument • Organization of Class and Class Environment: Instructor interview • Instructor Background: Instructor interview regarding demographic characteristics, training and experience in reading instruction

18. Field Study Measures: Program • ABE Program Operations: ABE Program Rating Form, Abt Associates Inc., which measures all components of program functioning • Program Characteristics: size, organizational structure, funding, number of staff, other characteristics

19. Field Study Data Collection Design • Learner: Pre-post design--fall (pre) and spring (post) of each year of data collection • Instructor: • Treatment instructors: multiple observations and documentation of teaching; interview after each observation • Control and comparison instructors: one observation per year; interview after each observation • Program: pre-post site visit (pre during year 1 of data collection, post during year 2 of data collection)

20. Field Study Progress to Date and Schedule • Comparison Group (7 programs participating): • Cohort 1 (47 learners) pre-tested; post-test in May 2004 • Year 1 class observations and interviews with program staff completed in May 2004 • Treatment and Control Groups (14 programs): • Majority of programs recruited • Random assignment in June • Treatment instructors will be trained in intervention curriculum during the summer • Data collectors will be trained during late summer