Identifying Research-based Practices for RtI : Scientifically-based Reading Instruction

Identifying Research-based Practices for RtI: Scientifically-based Reading Instruction Janet S. Twyman, Ph.D., BCBA Presented at the 2nd AnnualEvidenced-Based Practices Summitof the Wing Institute Berkeley, CA April 26, 2007

Historical Perspective: Reading in America • After a century and a half of universal public education, and despite the highest per-pupil expenditure on public elementary and secondary education in the world, 40 percent of U.S. fourth-graders are reading below the minimally acceptable level (NAEP 2006). • For minority students in inner-city schools, the reading failure rate is 65 percent. • Children who don’t read by fourth grade almost always fall behind in all other subjects, frequently require special education programs, and, as adults, have higher rates of drug addiction, incarceration, and welfare dependency.

Legislation to Avert our National Reading Crisis The No Child Left Behind Act of 2001 commits the nation to ensuring that every child in the country will learn to read by the end of third grade. Within NCLB, Reading First was first authorized at more than $6 billion to apply scientifically based reading research to strengthen children’s reading skills from kindergarten through third grade. Yet, at $1 billion per yearit accounts for only 2 percent of federal education spending.

Reading First – An Overview • As part of NCLB, it mandates that the best scientific research should guide the teaching of reading. • It supports research-based methods to ensure that every child in public schools reads at or above grade level by third grade. • The essential components of reading instruction are defined in the bill as explicit and systematic instruction in phonemic awareness; phonics; vocabulary development; reading fluency, including oral reading skills; and reading comprehension strategies.

Reading First – What it Means to Schools • Preparing teachers through professional development and other support, so they can identify specific reading barriers facing their students and have the tools to effectively help their student to learn to read. • Selecting or developing rigorous diagnostic reading assessments that document (1) effectiveness in improving students' reading and (2) the schools accountability for their results. • Selecting or developing effective instructional materials, programs, and strategies to implement methods that have been proven to prevent or remediate reading failure (including newly identified Response to Intervention Initiatives). • Establishing reading programs for students in grades kindergarten through 3 that are grounded in scientifically-based reading research (SBRR), in order to ensure that every student can read at grade level or above by the end of the third grade.

What the Law says is Scientifically-Based Reading Research The term `scientifically based reading research' means research that-- (A) applies rigorous, systematic, and objective procedures to obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and (B) includes research that-- (i) employs systematic, empirical methods that draw on observation or experiment; (ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn; (iii) relies on measurements or observational methods that provide valid data across evaluators and observers and across multiple measurements and observations; and (iv) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review. Source: H.R.1. 2001

“Scientifically-based” - Controversies Abound There little agreement about what constitutes “evidence.” • NCLB permits both quantitative and qualitative evidence (without clearly specifying the types of questions best answered by each approach). • Only small number of interventions used today have been empirically evaluated by some peer-reviewed standard. • Evidence-based interventions are less likely to be used than interventions for which there is no evidence or evidence showsa lack of impact (Kazdin, 2000). • In many instances educators/practitioners are not aware of evidence-based interventions (Kratochwill, Albers & Shernoff, 2004). Source: Detrich, Keyworth, & States (2007). Evidence-based Education: It Isn’t as Simple as You Might Think.

What the education field refers to as “Research-Based” Current uses of the term research–based in federal documents and journal articles (referred to as “evidence” when applied to early reading programs): • the program contains elements that research suggests are effective (see for example Simmons & Kame’enui, 2003) • pretest vs. posttest or simple comparison studies have provided some evidence of effectiveness for an instructional program, or • the program has undergone some form of scientifically controlled study, often involving randomized control groups (Coalition for Evidence–Based Policy, 2003)

However, …these uses fail to distinguish between the scientific development of a program, and the scientific evaluation of outcomes after a program has been developed.

Types of “Research” to consider when evaluating programs: • Research-InformedA program based upon best practices identified by research or contain elements that other research suggests are effective.-No requirement that the program itself be scientifically designed. • Research-Filtered (Summative Evaluation)A program, regardless of how it was designed, measured against an alternative form of instruction, or at times, no instruction at all. -No requirement that the program itself be scientifically designed. • Research-Guided (Formative Evaluation)A program that has been scientifically designed and tested during its development (most often guided by previous research results). In the research–guided approach, formative evaluation is intertwined into the instructional design protocols, and at its most thorough influences program development through iterations of testing, revising, and testing again.

How does one know what “evidence” to use? • Evidence is more than ONE study showing support. • Involves a body of knowledge, specifying: • the quality of evidence • the quantity and type of replication of effects • Level or rigor of evidence falls along a continuum. • Sufficient Threshold: • Evidence must be of a specific quantity and quality before a program should be considered evidence-based. • Hierarchy of evidence approach: • Rely on the best available evidence to date. Source: Detrich, Keyworth, & States (2007). Evidence-based Education: It Isn’t as Simple as You Might Think.

Research-Guided and Informed Current “Gold Standard”High QualityRandomized Controlled Trial Meta-analysis (systematic review) Single Case Designs Repeated Systematic Measures Research-Guided Semi-Randomized Trials Single Case Replication (Direct and Parametric) Well-conducted Clinical Studies Research-Filtered Threshold of Evidence Convergent Evidence Uncontrolled Studies Research-Informed Expert Opinion Consensus of Experts Various Investigations General Consensus Common Wisdom or Lore Personal Observation Single Study Continuums of Evidence Quantity of the Evidence Quality of the Evidence Program Development

An Example:How states determined their Reading First programs:

States Report Difficulty in Determining Evidence-based Reading Programs

Are we preparing our teachers? How can we expect oureducators to understand evidence-based instructionin reading, if our teacher preparation institutions aren’t teaching reading science? 85% earned a “failing” grade, even though a passing grade was possible if a professor devoted less than 20 percent of the lectures to the science of reading. All 5 4/5 3/5 2/5 1/5 None ofcomponents the components Source: National Council on Teacher Quality

When faced with “evidence” …ask at least three questions: • Is the evidence valid?Where does the evidence fit within the continuum? Could it be biased? • Is it important?Are the findings to relevant your situation? What is the significance of the outcome compared to other options? • Is it applicable to the learner in front of me?Are the participants and procedures in the original research comparable to your learner and setting?

Sources for “Evidence” http://www.whatworks.ed.gov

Assistance in determining evidence:

Florida Center for Reading Research www.fcrr.org/FCRRreports Provides one of the most comprehensive lists of reviews of reading programs http://www.fcrr.org

Oregon Reading First “Curriculum Review” leads to the Consumer's Guide to Evaluating Reading Programsan evaluation of various reading programs. http://oregonreadingfirst.uoregon.edu

Big Ideas in Beginning Reading http://reading.uoregon.edu/curricula/

Promising Practices Network http://www.promisingpractices.net/programs.asp

National Institute for Literacy http://www.nifl.gov/cgi-bin/pfr/search.cgi

California Learning ResourceNetwork Logging in provides access to evaluations of reading programs http://www.clrn.org

Interventions in School Psych. “Projects” leads to: Procedural and Coding Manual for Identification of Evidence-Based Interventions http://www.sp-ebi.org

Other Interventions http://nrepp.samhsa.gov/index.htm

IRIS Center – Star Legacy Modules Interactive Tutorials on:ACCOMMODATIONS BEHAVIOR COLLABORATION DIFFERENTIATED INSTRUCTION Response to Intervention Reading Strategies DISABILITY DIVERSITY

Scientifically Based Reading Research (SBRR)http://www.headsprout.com/aol/teachers.cfm

Reading FirstTechnical Assistance Centers Western Regional Reading First Technical Assistance Center (WRRFTAC) Center on Teaching & LearningUniversity of Oregon (Eugene) (541) 346-1608 Central Regional Reading First Technical Assistance Center (CRRFTAC) Center for Reading & Language Arts The University of Texas at Austin(512) 232-1901 Eastern Regional Reading First Technical Assistance Center (ERRFTAC) Florida Center for Reading ResearchFlorida State University (Tallahassee) (850) 644-9352

Wing Institute Home Page

What does this mean for real educators, real students, real classrooms? • Find and use programs with rigorous levels of research • Programs furthest along the continuum of empirical evidence of effectiveness (quantity, quality, and type of program development) • Access available public resources reviewing “evidence” • Learn from the community of available tools and tutorials • Use and be responsive to data from your own students, classroom, school • CCCROD • CBM • Progress Monitoring • DIBELS* • AIMSweb*

Source: Need for Research into Practice • A key outcome of evidence-based education is for scientifically proven interventions to be adopted and successfully implemented at the “practice” level. • In order for research to translate into practice, interventions must be efficacious (have a foundation in rigorous scientific research) as well as effective (address the necessary social influence factors to ensure the desired outcomes).

RtI facilitates the use of evidence and research-based programs • RtI framework encourages a system for identifying: need, treatments, measuring progress and outcomes, evaluating teaching practices, and revising whenever the data indicate to do so. • RtI is integral to knowing what is working and what is not, for each individual child.

Relevance for Special Education students • For some students require the modification of the general education reading curriculum, in terms of content or instructional approach. • Other students may require general education classroom reading instruction to be supplemented with an alternative, more intensive program. • Yet other students may require a complete replacement reading program, including other publicly available materials, or “teacher-devised” curricula.

Suggested Principles Regarding Reading Instruction for Special Education Students • Address all areas of the reading process as identified by reading research. • Identify, early, students demonstrating difficulty learning to read. • Design a tier (continuum) of intervention services prior to referral to special education. • Include in each special education student’s individual plan of instruction carefully selected curriculum and instructional techniques matched to student needs. • Base instructional plans on initial and ongoing assessment that reflects research-based best practice, including a historical analysis of the student’s reading instruction, consideration of the student’s specific characteristics and knowledge of the student’s deficits and strengths. • Use assessment data to guide and inform instruction. • Vary the intensity of intervention (amount of time for reading instruction, the size of the instructional group, means of service delivery, and the nature of the selected reading curriculum) provided to individual special education students based on the level of need and progress made (or lack thereof). • Use technology, as appropriate, to enhance the development of reading skills and to ameliorate the effects of deficiencies in reading. Handout available in resource packet.

IMPORTANT CAVEAT • Don’t become formulaic;even with SBRR or RTI • Never lose sight of your own ongoing data collection and analysis

Identifying Research-based Practices for RtI : Scientifically-based Reading Instruction