Fourth Annual Summit on Evidence-Based Education Data-Based Decision Making The Achilles’ Heel of Evidence-Based Educat

1. Fourth Annual Summit on Evidence-Based Education Data-Based Decision Making The Achilles’ Heel of Evidence-Based Education

2. The Wing Institute independent, non-profit operating foundation promote evidence-based education policies and practices act as a catalyst to facilitate communication, cooperation and collaboration between individuals and organizations currently engaged in the study and implementation of evidence based education

3. The Wing Institute’s Strategic Vision Increased focus on Research to Practice in the Real-world in Real-time implementation and sustainability

4. The Wing Institute’s Strategic Vision Identify exemplars in evidence-based education research individuals policies models programs organizations Develop networks to facilitate collaboration Provide support for new ideas, research, and publications Facilitate cross-discipline cooperation

5. SUMMIT Participants: professional disciplines administrator advocate asst. superintendent behavior analyst case manager community outreach consultant curriculum designer executive director parent professor psychologist researcher school administrator school psychologist SEPLA director spec. ed director speech therapist superintendent teacher

7. SUMMIT Participants: Organizations 49 different organizations 11 consumer / parent / advocacy / public organizations parent groups regional centers CA DDS / DMH area boards advocacy groups service orgs 9 public school organizations school districts SELPAs county office of ed 13 private education organizations nonpublic schools nonpublic agencies private schools 12 universities education school psychology pediatrics social welfare special education 4 research & policy organizations

8. President Barack ObamaMARCH 9, 2009 “Promoting science… is about ensuring that scientific data is never distorted or concealed to serve a political agenda -- and that we make scientific decisions based on facts, not ideology.”

9. Secretary of Education Arne Duncan Wall Street Journal OpinionApril 22, 2009 “And when we can link student outcomes to teacher quality and teachers to their colleges of education, we can challenge these institutions to do the best possible job in preparing a new generation of outstanding educators. Without the data, we cannot even have the conversation, let alone discuss solutions.”

10. Obama Administration Education Policy Stimulus Bill has $ 100 Billion in new education funding Clues from the fine print of the economic stimulus law…four assurances that Governors must sign to receive billions in emergency education aid $ 5 Billion to reward states that make good on pledges $ 3 Billion for school turnarounds

11. Stimulus Package Assurances 1. Improve quality of standardized tests and raise standards that are internationally benchmarked (national academic standards) 2. Support struggling schools 3. Invest heavily in teacher and principal quality initiatives (assign most effective teachers equitably to all students, rich and poor) 4. Build robust data systems (including ability to link teachers to students and test scores--to distinguish between effective and ineffective teachers)

12. Avg 69% Avg 53% Avg 41%

13. % Agree / Strongly Agree students staff organization 74% data is collectedfrequently 31% 50% data measures something that is important 82% 58% 62% data is accurate 71% 57% 60% data is regularly and accurately analyzed by staff 47% 31% 42% data is used and useful in making decisions 58% 47% 47%

14. AVG = 51%

15. AVG = 77%

16. AVG = 55%

17. SUMMIT Outcomes STUDENT Data-Based Decision Making Examination of issues, obstacles, strategies for data collection, analysis and decision making. Frank Gresham STAFF Data-Based Decision Making Discussion of staff development, monitoring and feedback strategies to promote program integrity and accurate data collection. Dennis Reid ORGANIZATION Data-Based Decision Making Presentation of a model for identifying questions, data to collect, monitoring and feedback systems at the systems-level. W. David Tilly III

18. Summit Housekeeping Resource Packet (Flash Drive) Knowledge Network Schedule Breaks Lunch Time management (groups, report outs) Summit Feedback Survey Networking

19. Secretary of Education Arne Duncan Wall Street Journal OpinionApril 22, 2009 “We need a culture of accountability in America's education system if we want to be the best in the world. No more false choices about money versus reform, or traditional public schools versus charters. No more blaming parents or teachers. We need solid, unimpeachable information that identifies what's working and what's not working in our schools. Our children deserve no less.”

20. Thanks for Coming!

21. intuition opinion of others political considerations regulatory considerations fiscal considerations immediate contingencies conceptual framework past experience Other than data, what most influences your decision making: 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % Responses

22. too much data too long to collect poor data display time delay poor use of technology staff resistance data not accurate insufficient org. support insufficient resources insufficient training data doesn't measure desired outcomes inadequate data competing priorities What obstacles do you encounter in making effective data-based decisions? 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % Identified as Strong / Moderate Obstacle

23. The Ernie Wing Award for Excellence in Evidence-based Education The award is presented to an individual for outstanding achievement in the areas of the Evidence-based Education Road Map efficacy research effectiveness research implementation monitoring replicability and sustainability

24. The 2009 Ernie Wing Award for Excellence in Evidence-based Education In recognition of work in the development, implementation, and sustainability of Data-Based Decision Making Organizations and Cultures

25. Data-Based Decision Making Organizations and Cultures • Offer multiple programs for children with all levels of learning needs • Partner with public schools, universities, and other community agencies to help disseminate best practices • Engage in applied research to continually advance the science of education • Operate programs based on research-based, proven-effective practices • Use data-based decision making at all levels of the programs (student - teacher - organization)

26. Data-Based Decision Making Organizations and Cultures • Provide school cultures that are based on reinforcement rather than punishment • Offer programs that are “learner-centered”, customizing the educational programs to individual student needs • Get dramatic results • Have maintained this commitment to quality for over 25 years!!

27. The 2009 Ernie Wing Award(s) The Institute for Effective Education Ken Traupmann and Suzanne Fitch Morningside Academy Kent Johnson

28. Why do we care about sustainability? research to practice The ultimate goal of the “evidence-based movement” is make better use of research findings in typical service settings, to benefit consumers and society….

29. Research to Practice Evidence-based Education Research Replicability Sustainability Practice

30. Research Replicability What works? When does it work? Sustainability How do we make it work? Is it working? Practice Efficacy Research(What Works?) What works? • Currently, this is the most common form of published educational research. • Research conducted to identify promising practices • Characterized by: highly structured and controlled laboratory settings highly trained change agents carefully screened participants adequate resources close supervision.

31. SUMMIT Participants: Geography 11 different states California Nebraska Iowa New York Kentucky Ohio Louisiana Oregon North Carolina Utah Washington

32. Research Replicability What works? When does it work? Sustainability How do we make it work? How do we make it work? Is it working? Practice Implementation(How do we make it work?) • Translates effectiveness research to practice, from “general settings” to a “ particular setting” • Explicit, systematic process for analyzing and addressing the critical variables necessary for an intervention to be successfully adopted, implemented and sustained in a particular setting. • Analyzes the contingencies operating on various stakeholders in a particular practice setting and how they influence adoption and sustainability of an intervention.

33. Research Replicability What works? When does it work? Sustainability How do we make it work? Is it working? Is it working? Practice Performance Monitoring(Is it Working?) • To assure that the intervention is actually being effective must monitor the impact of the intervention in the setting (practice-based evidence). • Monitoring must occur: • student level (to ensure progress and be able to modify components of the intervention when necessary) • systems level (to be able to make systems level decisions and policy choices)

34. Research Replicability What works? When does it work? When does it work? Sustainability How do we make it work? Is it working? Practice Effectiveness Research(When Does it Work?) • Less precise, and common than efficacy research • Research conducted to answer questions about the impact and robustness of interventions when taken to scale in more typical practice settings. • Identifies critical variables necessary for successful translation of practice from research • characteristics of students, setting • leadership and instructors • resources, training available • culture, level of commitment

35. What do we know about “sustainability”? Research to Practice implementation and sustainability

36. Presenters Frank Gresham, Ph.D. • Professor in the Department of Psychology at Louisiana State University • Research-based instruction, technology and education, teacher training, and systems approaches to effective education • President-elect Association for Behavior Analysis W. David Tilly III, Ph.D. • Director of School Psychology training (University of Massachusetts) • Published widely in areas of CBM and educational assessment • Works with National Center for Student Progress Monitoring Dennis Reid, Ph.D. • Assistant Professor / Program Director School Psychology program (U Nebraska) • RtI Ad Hoc Committee for Nebraska Department of Education • Editor-elect of Journal for School Psychology

37. Data-Based Decision Making Organizations and Cultures • Offer multiple programs for children with all levels of learning needs • Partner with public schools, universities, and other community agencies • Engage in applied research to continually advance the science of education • Operate programs based on research-based, proven-effective practices • Use data-based decision making at all levels of the programs (student - teacher - organization) • Provide school cultures that are based on reinforcement rather than punishment • Offer programs that are “learner-centered”, customizing the educational programs to individual student needs • Get dramatic results (money back guarantees) • Have maintained this commitment to quality for over 25 years!!

38. The Wing Institute • independent, non-profit organization • founded in 2005 • mission: to promote evidence-based education policies and practices • experience and expertise operating "research based" special education services in “real world” settings • named after Ernie Wing

39. The Summit “Overview” Who we are Who you are Summit Outcomes & Strategies Survey Data

40. The Wing Institute’s Strategic Vision act as a “catalyst” to promote an evidence-based education culture by supporting individuals and organizations currently engaged in evidence based education

41. 30 years studying “research to practice” issues in education… from the “practice” side

42. The Wing Institute 1978 - 2004 Spectrum Center operated "research based" special education services in “real-world” settings… …provided a “laboratory” setting for longitudinal study of research to practice, implementation and sustainability

43. Obama Administration Education Nominees John Q. Eastin Nominated Director of Institute of Education Sciences Previously Director of Consortium on Chicago School Research (CCSR) at the University of Chicago CCSR conducts “hands-on” research to inform and assess policy and practice, (researcher-practitioner model) helping to build capacity for school reform

44. Summit Strategies: “pre-summit” READINGS “Commentary: Use of Evidence-based Interventions in Schools: Where We’ve Been. Where We Are, and Where We Need to Go” Hill Walker “Current Status and Future Directions of School-based Behavioral Interventions” Frank Gresham “Response to Intervention: Empirically Based Special Service Decisions from Single-Case Designs of Increasing and Decreasing Intensity” David Barnett, Edward Daly, III, Kevin Jones, and F. Edward Lentz, Jr.

45. Summit Strategies: RESOURCE PACKET Additional Articles on Data-Based Decision Making Systems Level Interventions Copies of Presentations

46. Summit Strategies: WORK GROUPS Practice evaluating “evidence” Identification of obstacles and opportunities for implementing RtI Generation of questions for speaker panel

47. Summit Strategies: PANEL DISCUSSION Panel Discussion with presenters Q&A from audience

48. Summit Strategies: NETWORKING Working with wide-range of education stake-holders at Summit Follow-up contacts

49. Summit Strategies: FOLLOW-UP WING KNOWLEDGE NETWORK FOLLOW-UP ACTIVITIES GENERATED FROM SUMMIT

50. Summit Strategies for Meeting Outcomes Pre-Summit Readings Pre-Summit Survey Resource Packet (flash drive) (KNOWLEDGE NETWORK) Presentations Work groups Panel Q&A Networking Knowledge Network