The National Special Education Research Agenda: Inside the Matrix

1. The National Special Education Research Agenda: Inside the Matrix

2. The National Center for Special Education Research (NCSER) in the Institute of Education Sciences (IES): But…What exactly is the Institute of Education Sciences? (And, what happened to OSERS and OSEP?)

3. IDEA Reauthorization in 2004 • Amended the The Education Sciences Reform Act of 2002 to establish the National Center for Special Education Research in the Institute of Education Sciences (IES) • Transferred responsibility for special education research and “Studies and Evaluations” (except for the Annual Report) from the Office of Special Education Programs to NCSER.

4. The Charge: Legislative Branch Research, statistical, and evaluation activities supported by the Institute shall “apply rigorous, systematic, and objective methodology to obtain reliable and valid knowledge … and present findings and make claims that are appropriate to and supported by the methods that have been employed.” (ESRA, 2002)

5. IES Goals • develop or identify programs, practices, policies, and approaches that enhance academic achievement and that can be widely deployed • identify what does not work and what is problematic, and thereby encourage innovation and further research • gain fundamental understanding of the processes that underlie variations in the effectiveness of education programs • develop delivery systems for the results of education research that will be routinely used by practitioners and the public when making education decisions

6. Organizational Structure Office of the Director Grover J. Whitehurst, Director National Board for Education Sciences Robert C. Granger, Chair Sonia Chessen, Executive Director Office of Communication and Outreach Mike Bowler, Director of Communications and Outreach Office of Administration and Policy Sue Betka, Deputy Director for Administration and Policy Office of Science Andrew White, Deputy Director for Science Office of Information Technology Gerald Malitz, Chief Information Technology Officer

7. Organizational Structure Office of the Director Grover J. Whitehurst, Director National Board for Education Sciences Robert C. Granger, Chair Sonia Chessen, Executive Director National Center for Education Statistics Mark Schneider, Commissioner National Center for Education Research Lynn Okagaki, Commissioner National Center for Education Evaluation and Regional Assistance Phoebe Cottingham, Commissioner National Center for Special Education Research Edward J. Kame’enui, Commissioner

8. The National Center for Special Education Research (NCSER) in The Institute of Education Sciences (IES):In Appreciation of Hartley’s First Law

9. Hartley’s First Law You can lead a horse to water, but if you can get him to float on his back, you’ve got something. Bloch, A. (1982). Murphy’s law and other reasons why things go wrong. LA: Price/Stern/Sloan Publishers, Inc.

10. Five Organizing Principles for National Center for Special Education Research in the Institute of Education Sciences

11. NCSER Organizing Principles 1. The “science” must come first--

12. Ode to Feynman This phenomenon of having a memory for the race, of having an accumulated knowledge passable from one generation to another, was new in the world. But it had a disease in it. It was possible to pass on mistaken ideas. It was possible to pass on ideas which were not profitable for the race… Then a way of avoiding the disease was discovered… And that is what science is: the result of the discovery that it is worthwhile rechecking by new direct experience, and not necessarily trusting the race experience from the past (p. 185). Richard P. Feynman (1999). The pleasure of finding things out. Cambridge, MA: Perseus Publishing.

13. “We’ve done well…now there’s an opportunity to really be amazing--to be amazing as a business, to be amazing in the positive impact that we have on society. But we have to do some things a little bit differently to be as amazing as we hope we can be.” Steve Ballmer, CEO, MicrosoftBusiness Week, June 17, 2002 (http://www.business week.com/magazine/content/02_24/b3787001.htm) Invoking the Ballmer Standard

14. NCSER Organizing Principles 1. The “science” must come first--and it must be the best, perhaps even “amazing.”

15. Methodological & Statistical Issues • Traditional issues: Highly nested nature of student performance in complex systems that provoke complex interactions among range of variables (familial, social, pedagogical, curricular, organizational) compounded by heterogeneity of student characteristics/performance; comorbidity. • Quantitative synthesis (meta-analysis) of single-case research to identify effective educational interventions; • Statistical power for random assignment evaluations of programs and interventions; etc.

16. Evidence-based Education • Using the best available empirical evidence in making decisions about education -- Particularly for students with disabilities

17. IES NCSER + Practice (TA & D) Simple Formula 1. R - TA & D = Irrelevant 2. TA & D - R = Irresponsible 3. R + TA & D = Relevant & Responsible What Does This Mean? • Levels of evidence process & mechanisms

18. Levels of Evidence on What Works • Meta-analyses of high quality evidence • Experiments and well designed quasi-experiments using WWC standards (including small n designs) • Statistical modeling of correlational and longitudinal data • Best practice studies with matching and contrastive analysis • Expert opinion supported by conceptual models and generalizations from high quality research on related topics • Simple correlational studies, case studies, pre-post studies, and best practice studies w/o matching

19. What Can CEC Do? • Establish a “Scientific Advisory Council” to anchor the primacy of science in CEC • Advocate & insist upon rigorous scientific standards in special education research + TA & D (and within organization) • Adopt levels of evidence to adjudicate/promote research-to-practice translation • Launch a public awareness campaign to inform parents about importance of science

20. President’s FY 2006 Education Final Appropriations (in millions of dollars) • Title I Grants to LEAs = $12,713 million (largest portion of NCLB programs) • Reading First/Early Reading First = $1,132 million • Research in special education and studies and evaluation = $82 million (0.11% of total ED monies) • Total Education Appropriation = $71,545 million Reference: http://www.ed.gov/about/overview/budget/budget06/06action.pdf

21. Comparison Agency Defense Budget Authority = $419,341 Homeland Security Authority= $34,152 Secret Service Agency = $1,200 Environmental Protection Agency = $7,571 Total Federal Budget = $2,568 (in billions of dollars) Comparative % for NCSER ($82 million): 0.02% monies authorized as compared to DOD 0.24% of monies authorized as compared to DHS 6.83% of monies authorized as compared to Secret Service Agency 1.08% of monies authorized as compared with the EPA 0.003% of total monies authorized Comparison of FY2006 Appropriations with other Federal agencies (millions of dollars) Reference: http://www.gpoaccess.gov/usbudget/fy06/browse.html

22. Federal Research and Development Spending (Proposed: FY 2006) • Total Across Federal Agencies = $132,304 million • Federal agencies include: DOD, HHS, NASA, DOE, NSF, USDA, DHS, DOC, DOT, DVA, DOI, EPA, and Other • “R&D” is characterized as “basic research, applied research, development, R& D equipment or R&D facilities”. • NCSER Appropriation = $81.7 million (0.06% of R&D monies) • Reference: http://www.whitehouse.gov/omb/budget/fy2006/budget.html

23. NCSER Organizing Principles 2. Because the special education “problem space” is infinitely broad and complex, we must invest wisely and strategically.

24. To do so requires going “inside” the Special Education Matrix What Special Education Matrix?

25. Features of NCSER Research Matrix NCSER Statutory responsibilities: 18 Disability Statutory categories: 12+ Total Research Matrix cells: 18 x 12 = 216

26. Matrix of Disability Category by NCSER Duties

27. Disability Category: Specific Learning Disabilities Age—Growth & Development x Time Content (1) Infants/ Toddlers (2) Pre-school (3) Kinder-garten (4)-(6) Grades 1 through 3 (7)—(16) Grades 4 through 12 (17)—(25) School to post-school, post-secondary, etc. Early Childhood School Age Transition (1) Speech and Language Development (2) Beginning Reading (Learning to Read) (3) Content Reading (Reading to Learn) (4) Etc. NCSER Evidence Map: Just One of 216 Cells

28. FY 2006 NCSER Research Programs • Reading & Writing Special Education Research • Math & Science Special Education Research • Special Education Teacher Quality – Reading/Writing • Special Education Teacher Quality – Math/Science • Early Intervention and Assessment for Young Children with Disabilities

29. FY 2006 NCSER Programs (cont’d) • Assessment for Accountability Special Education Research • Individualized Education Programs Research • Serious Behavior Disorders Special Education Research • Secondary and Post-Secondary Outcomes Special Education Research • Language and Vocabulary Development Special Education Research

30. FY 2007 Grant Competitions • Early Intervention, Early Childhood Special Education, and Assessment for Young Children with Disabilities • Mathematics and Science • Reading, Writing, and Language • Serious Behavior Disorders • Assessment for Accountability

31. FY 2007 Grant Competitions • Individualized Education Programs and Individualized Family Service Plans • Secondary and Transition Services • Quality of Teacher and Other Service Providers for Students with Disabilities • Autism Spectrum Disorders • Response to Intervention

32. Research Grant Program Goals Goals differ by topic area, and include: • Goal 1: Identifyingpromising practices. • Goal 2: Developing and testing new approaches. • Goal 3: Conducting efficacy and replication trials. • Goal 4: Conducting large scale evaluations. • Goal 5: Developing and validating assessments.

33. NCSER Organizing Principles 3. Research in “special education” is at the heart of the National Center for Special Education Research (NCSER) in IES.

34. Public Law 108-446—December 3, 2004 118 STAT. 2647 - 108th Congress USC 1401 Sec. 602. DEFINITIONS • “(29) SPECIAL EDUCATION.—The term ‘special education’ means specially designed instruction, at no cost to parents, to meet the unique needs of a child with a disability, including— • “(A) instruction conducted in the classroom, in the home, in hospitals and institutions, and in other settings; and • “(b) instruction in physical education.

35. NOT special institutions teaching strategies curricular programs special support special learning general programs Design = Architecture of information Individualized Education Program (IEP) AND “Related Services” Families Specially Designed Instruction

36. NCSER Organizing Principles 4. Building the research capacity of the field is not a “Lone Ranger” endeavor: Collaboration and consultation are essential.

37. The Charge: Customers • Make research more practical • Fill gaps in knowledge • Provide digested and vetted findings • Consider products for end users • Consider expanding current IES research grammar and goal structure

38. Office of Special Education and Rehabilitative Services (OSERS) Dr. Troy Justesen, Deputy Assistant Secretary Dr. Alexa E. Posny, Director, Office of Special Education Programs (OSEP) Dr. Lou Danielson, Director of Research to Practice Division John Hager, Assistant Secretary

39. Ode to Boren’s First Law 5. Good science is good “business” for everyone; but good science is not enough. Results must lead to the engineering of high quality performance and delivery systems.

40. “Why the Global Economy Is Here To Stay” Jeffrey E. Garten, Dean, Yale School of Management (Business Week, March 23, 1998, p. 21) Delivering some 2.8 million packages in 210 countries each day, FedEx is pushing globalization faster and deeper…because it is using information technology to reengineer its clients’ worldwide supply and distribution systems…FedEx can…electronically track where any shipment is at any given moment, and it can guarantee on-time delivery.

41. “Cream of the Crop,” Alison Peacock. Horizon Air, April, 1998, p. 12-17. “A computerized system keeps track of how much milk each cow gives each time. Obrist can look up any cow on his computer and track her milking yields for weeks at a time, noting at a glance when she is at peak production or if her health is ailing” (p. 13).

42. Engineering Systems IT Systems (including digital technology) that permit, for example: • the large scale, real time management of a range of student performance data at the district, school, and classroom levels, including the real time management of individualized education programs (e.g., FedEx; Dairy Farmers); • greater and faster accessibility to the full range of print materials; • accountability and information management systems linked to student performance and results

43. Five Harsh Realities to Hartley’s First Law

44. Harsh Realities • We have more cells than research dollars. Most cells are either empty or partially filled. None of the cells are completely filled. Where should NCSER make investments for the short and long term? What organizing principles and values should guide these decisions? • Space and time is at least three-dimensional: Investments must reflect a long-term horizontal view (breadth) with vertical short-term investments (depth) that vary over time, complimented with diagonal investments when possible/feasible (depending on quality of research from field).

45. Harsh Realities (Cont’d) • What is the capacity of the field to deliver the research “goods”? The quality of the research infrastructure and capacity is essential to good science and research. Needed: Short and long-term investments in building professional capacity that is interdisciplinary, rigorous, and content intensive in special education as a primary discipline.

46. Harsh Realities (cont’d) • What is the capacity of the U.S. Department of Education and the Institute of Education Sciences (IES) to deliver the goods? • What are the mechanisms for obtaining input from the field—individuals and parents/ caregivers of children and individuals with disabilities, professional organizations, stakeholders and the general public—on the research investments?

47. Stockmayer’s Theorem If it looks easy, it’s tough. If it looks tough, it’s damn well impossible. Bloch, A. (1982). Murphy’s law and other reasons why things go wrong. LA: Price/Stern/Sloan Publishers, Inc.

48. The home of evidence-based education