Bucks County Autism Support Coalition What is Evidence-Based Practice? Why is it important?

1. Bucks County Autism Support CoalitionWhat is Evidence-Based Practice?Why is it important?

2. Today’s Goals • Why we rely on science and technology • Defining EBP • Implementing EBP • Identifying EBP • The Future of EBP

3. My background • As a father • As a clinician • As a teacher • As a researcher • As a program leader

4. WHY SCIENCE & TECHNOLOGY?How these have improved our lives • The Ghost Map • Brunelleschi’s Dome • The Brooklyn Bridge • EIBI

5. WHY SCIENCE & TECHNOLOGY? Why do we need EBP? • Extraordinary and unfounded claims of success re Autism are made every day • Science can inform decisions about individual treatment • Maximize selection of effective treatments • Minimize selection of ineffective treatments. • Avoid harmful treatments. • Align treatment options within an individual across disciplines.

6. WHY SCIENCE & TECHNOLOGY? Why do we need EBP? • Science can also drive systems changes • Set training priorities • Identify levels of support • Help to set policy • Problem: Now everybody is claiming that everything is scientifically-based!

7. WHY SCIENCE & TECHNOLOGY? Non-Scientific Best Practices? • Basic legal and ethical standards? Necessary but insufficient for EPB • Cultural values? Be careful when these collide with science • Is inclusion values-based or data based? • Can we ever eliminate the need for seclusion and/or restraint

8. DEFINING EBPWhat are Evidence-Based Practices? 1. An intervention resulting in an outcome that is • Clinically / educationally significant AND • Superior to that obtained otherwise 2. That specifies (a) methods used, (b) treatment target, and (c) who will benefit 3. That is demonstrated via a series of outcome studies that are at least well-controlled and peer-reviewed, if not independently replicated

9. DEFINING EBP1. Significant outcomes • Must result in a specific and immediate improvement in day-to-day functioning • Improved social, communication, or adaptive skills, or decreased behavior problems • High Impact Outcomes help your child do better TODAY across settings • Be wary of improvements that have “theoretical” but not functional significance

10. DEFINING EBP 2a. Specific methods • We must know the specific method(s) if we are to replicate the child’s treatment • Proper level of analysis yields predictable, significant & replicable results for a designated target XApproaches: A discipline (ABA), or general theory or model (Developmental)? Far too broad ? Programs: A service delivery model that may specify a method, curriculum, population, resources, and /or setting (EIBI)? Too broad to easily replicate  Methods: A specific technique that may be used for a broad range or narrow range of targets

11. DEFINING EBP 2b. Specific targets • A close match between the target in the study & the goal for your child makes it more likely you will achieve similar results • Outcomes identified by reviewers can be relatively broad (social skills, communication skills, etc.)

12. DEFINING EBP 2c. Specific populations • A close match between the children in the study and you child makes it more likely you will achieve similar results • Populations identified by some reviewers are also broad • Children with / without ASD • Children with / without Intellectual Disability • What are the ideal outcomes? • Intervention B is effective in teaching Skill C to Child with characteristics D, E, F and in yielding Outcome G

13. DEFINING EBP 3. Outcome vs other research • Clinical anecdotes & parent/provider surveys are not considered outcome research • Basic research: Important in testing models but not outcomes • Published reviews of outcome research? Rarely apply a clear standard OR are definitive enough • Exception might be position papers of professional organizations

14. DEFINING EBP 3. Well-controlled outcome studies • Outcome research? A specific study directly testing the effectiveness of a specific intervention in human subjects • Well-controlled? Meets minimal standards of quality established via peer review • Peer review publication provides an additional level of quality control • But it isn’t it enough because reviewers are not equally rigorous in evaluation of quality

15. DEFINING EBP 3. Standards for Outcome studies • Recent reviews with overlapping criteria • National Autism Center (2009) • National Professional Development Center for Autism at UNC (2010) • Reichow, Doehring, Cicchetti, & Volkmar, (2010) • All of these • Accept that group designs AND single subject designs are acceptable • Focus on specific methods, and try to identify for which skills these are effective

16. DEFINING EBP 3. Independent replication • Independent replication: Ultimate test of methodology • Means that others can achieve same results • Protection against fraud • Helps to ensure that some other unidentified factor were not responsible for success

17. IMPLEMENTING EBPEnsuring Treatment Fidelity • To implement interventions reliably you usually need • Training manual • Training program with fidelity checks and coaching • Places to start • NPDC Evidence briefs: one way to start • Video demonstrations via OCALI’s Autism Internet Modules project • Other trademarked training programs deserve careful attention

18. IMPLEMENTING EBPMatching Treatment Intensity • How often must the intervention be delivered to begin to see effects? • What kinds of resources do you need to implement intervention faithfully • Specialized staff and staff supervision • Specialized materials and settings • Non-specific treatment variables are rarely considered in evaluating programs • Data suggest that these variable are very important

19. IMPLEMENTING EBPA culture of data-based decision making • In evaluating individual treatment, do we • Set measurable objectives • Gather data regarding progress towards these objectives • Analyze the data at regular intervals • Make treatment decisions based on – or at least informed by – these results • Are prepared to be wrong! • The best techniques will fail if you do not do all of the above

20. IMPLEMENTING EBP Costs • What are the costs? • Hiring and training staff • Specialized materials and settings • Opportunity costs: Investing energy and effort here instead of eslewhere • Do the benefits outweigh the costs? • How will this result in a real and significant improvement in their life?

21. IMPLEMENTING EBPInterventions which lack data? • Don’t judge too quickly! Don’t dismiss all interventions that lack data! • Remarkably few interventions are research-based • All are well-intentioned, so build bridges! • Be open to all possible lessons & future directions • Too new to be tested? It is realistic to expect published, peer-reviewed data within 2-3 years of initial claims • But choosing such an intervention, over a cost-effective evidenced-based practice may not be defensible

22. IMPLEMENTING EBP Questions parents should ask • If someone makes a claim that a practice is evidence based? • 1. Show me the data (peer reviewed outcome research)! • One really good study or several OK studies from different researchers might make the practice promising • 2. Are the study(ies) good / high quality? • 3. Are these specific methods, not broad programs?

23. IMPLEMENTING EBP Questions parents should ask • 4. Is the treatment target a priority? • 5. Does the study population and targets match my child and my goals? • 5. Can we achieve the same treatment fidelity and intensity? • Implement, carefully evaluate costs and outcomes, and change course based on the data

24. IMPLEMENTING EBP Questions practitioners should ask • When setting overall training priorities, practitioners and leaders should focus on • 1) Behavior or skill deficits that A) Occur most often B) Have the greatest impact

25. IMPLEMENTING EBP Questions practitioners should ask • 2) Cost-effective interventions that A) Have a rapid impact B) Can be implemented reliably C) Are less intensive D) Are clearly effective relative to other interventions • Few interventions will fit all of these criteria • All must be framed within client values & preferences

26. IMPLEMENTING EBP Questions practitioners should ask

27. IDENTIFYING EBPReducing Problem Behavior

28. IDENTIFYING EBPReducing Problem Behavior

29. IDENTIFYING EBPReducing Problem Behavior

30. IDENTIFYING EBPIncreasing basic communication

31. IDENTIFYING EBPIncreasing basic communication

32. IDENTIFYING EBPTeaching basic social skills

33. IDENTIFYING EBPUsing EBP to drive treatment • Unless you can quickly show that another technique yields better results, you should always turn to EBP first • EBP: Prioritize ABA-based, positive & proactive interventions informed by functional assessment for reducing challenging behavior before • Relying solely on Medication • Using sensory integration • Using Alternative/complementary interventions (e.g., GFCF diets)

34. THE FUTURE OF EBPNew approaches • Identify more specific, individualized goals from an EBP using patterns of evidence • Consistent evidence that PECS can be mastered relatively quickly, and that improvements are correlated with complexity of speech and other social-communicative behavior • Choosing methods based on child characteristics • To increase Joint Attention, use RPMT when JA is already established, and PECS when it is not

35. THE FUTURE OF EBPEvidence-Based Policy • Policy Program Development • Needs Assessment

36. THE FUTURE OF EBPA new focus on technology • Policy Program Development • Needs Assessment • New Program

37. THE FUTURE OF EBPIdentifying new services • Policy Program Development • Needs Assessment • New Program