Thinking Critically About Autism Treatments

1. Thinking Critically About Autism Treatments

2. What Interventions Should We Use? The Right to Effective Behavioral Treatment ABA Task Force (1987) Published in JABA (1988) http://www.abainternational.org/ABA/statements/treatment.asp Individuals who are recipients of treatment designed to change their behavior have the right to: A therapeutic environment Services whose overriding goal is personal welfare Treatment by a competent behavior analyst Programs that teach functional skills Behavioral assessment and ongoing evaluation The most effective treatment procedures available

3. An individual is entitled to effective and scientifically validated treatment; • In turn, the behavior analyst has an obligation to use procedures demonstrated to be effective The Most Effective Treatment Procedures Available

4. Section 2.09 Treatment Efficacy • http://www.bacb.com/consum_frame.html • The behavior analyst always has the responsibility to recommend scientifically supported most effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society. • Clients have a right to effective treatment (i.e., based on the research literature and adapted to the individual client). • Behavior analysts are responsible for review and appraisal of likely effects of all alternative treatments, including those provided by other disciplines and no intervention. Behavior Analyst Certification Board (BACB) Guidelines for Responsible Conduct

5. No Child Left Behind Act (2001) • Interventions based on scientific research should be used to improve educational outcomes • Individuals with Disabilities Education Improvement Act (IDEA, 2004) • Special ed services in IEP are to be based on “peer reviewed research to the extent practicable” Education Law

6. How Do Parents Choose a Treatment for Their Child with Autism? Recommendation by pediatrician or other doctor School Other parent Internet, book Do you think these sources reliably recommend interventions based on the objectivity of the evidence?

7. Metz, Mulick, and Butter (2005): Google search - autism and treatment = 65 distinct interventions sold as effective for treating autism • Telepathy, injection of sheep stem cells, thyme, swimming with dolphins • Fad: a procedure, method, or therapy that is adopted rapidly in the presence of little validating research, gains wide use or recognition, and then fades from use – usually in the face of disconfirming research, but often due to the adoption of a new fad • In autism treatment, fads tend to be harmful • Waste time • Waste money • Falsely raise hopes and expectations • Distract from effective treatments • In some cases, harm children and families Fads in Autism Treatment

8. Early 1900s • Wilhelm Von Osten, a high school math teacher and horse trainer, taught Hans to • Add, subtract, multiply, divide • Work with fractions • Tell time • Read, spell • Hans tapped out the answer • Von Osten game him a carrot when he was correct Clever Hans

9. Team of 13 people came to see if it was true – including a circus trainer, zoologist, vet, politician • Psychologist Oskar Pfungst used research methods to test Hans • When Von Osten didn’t know the answers, Hans didn’t either! • Almost imperceptible body movements cued Hans

10. Set of ideas based on theories put forth as scientific when they are not scientific -The Skeptic’s Dictionary (http://skepdic.com/pseudosc.html) • A collection of related beliefs about the world mistakenly regarded as being based on scientific method or as having the status that scientific truths now have - The Oxford English Dictionary (2nd edition) • Tries to lend credibility to beliefs, speculations, and untested assumptions by cloaking them in the accouterments of science – for example, by using scientific-sounding jargon, getting endorsements from individuals with apparent scientific credentials, perhaps even by using numbers or graphs. -Green (1999) Pseudoscience

11. High "success" rates are claimed. • Rapid effects are promised. • The therapy is said to be effective for many symptoms or disorders. • The "theory" behind the therapy contradicts objective knowledge (and sometimes, common sense). • The therapy is said to be easy to administer, requiring little training or expertise. • Other, proven treatments are said to be unnecessary, inferior, or harmful. • Promoters of the therapy are working outside their area of expertise. • Promoters benefit financially or otherwise from adoption of the therapy. • Testimonials, anecdotes, or personal accounts are offered in support of claims about the therapy's effectiveness, but little or no objective evidence is provided. • Catchy, emotionally appealing slogans are used in marketing the therapy. • Belief and faith are said to be necessary for the therapy to "work." • Skepticism and critical evaluation are said to make the therapy's effects evaporate. • Promoters resist objective evaluation and scrutiny of the therapy by others. • Negative findings from scientific studies are ignored or dismissed. • Critics and scientific investigators are often met with hostility, and are accused of persecuting the promoters, being "close-minded," or having some ulterior motive for "debunking" the therapy. Pseudoscientific Therapies: Some Warning Signs(adapted from the American Arthritis Foundation)*originally printed in Science in Autism Treatment, Spring 1999http://www.asatonline.org/resources/articles/evaluate.htm

12. All-Time Favorite Pseudoscientifc “Therapy” • http://www.autismlink.com/listing/adam_technology_autism_program_at_galaxy_wave_group • http://www.randi.org/jr/050704your.html

13. They’re presented as relatively easy and with immediate effects by people who appear warm, sincere, and attentive • Best treatment is deemed distasteful, is hard to get, is expensive, requires a lot of training, is hard to understand • Alternate treatments are supported by popular culture, “feel” right, seem to make sense • Professionals or other people recommend them • Most professionals are not trained how to evaluate treatments • Autism treatment is a commercial enterprise Why Do Ineffective/Unproven Treatments Become Fads? (Vyse, 2005)

14. Parents are in a vulnerable position • Doing SOMETHING feels better than doing nothing • Grieving process • Avoidance of guilt • Urgency • Qualities that make parents dedicated and enthusiastic make them vulnerable to accept claims without close scrutiny • Nature of Autism • Cause is unknown • Autism is mysterious – myth of the “hidden inner child” Why Do Ineffective/Unproven Treatments Become Fads? (Metz et al., 2005)

15. Starting with Knowledge Knowledge is information about how things work Information is only useful if it is accurate (“valid”) Unfortunately, lots of knowledge sources have errors and subjective biases How do we know what knowledge (information) is accurate? 15

16. Methods of Acquiring Knowledge Tenacity – information is accepted as true because it has been believed for a long time Intuition – information is accepted as true because it “feels” like it’s true Authority – information is accepted as true because it comes from authorities/ experts We all know…sometimes knowledge from these sources is accurate and sometimes it’s NOT. Should we discount knowledge from these sources? No, use them as a guide – view this information tentatively until you have better evidence. Should we use these sources to obtain knowledge About psychology? About how human behavior works? About how to provide the best treatment for our clients? Well, what are other choices?

17. Methods of Acquiring Knowledge Rationalism – obtaining knowledge via logic or reasoning. It’s a way of establishing truth in the absence of evidence. Empiricism – obtaining knowledge through observation or direct experience However, science and scientists are imperfect… Be skeptical of research results (be a wise consumer!) Check out the credentials of the source Check out the funding source and possible agenda Example: MMR vaccine and autism

18. What is Science? An attempt to find order and regularities in the world

19. “Science is an attempt, largely successful, to understand the world, to get a grip on things, to get hold of ourselves, to steer a safe course. Microbiology and meteorology now explain what only a few centuries ago was considered sufficient cause to burn women to death.”-Carl Sagan, The Demon-Haunted World

20. What truly marks an open-minded person is the willingness to follow where evidence leads. The open-minded person is willing to defer to impartial investigations rather than to his own predilections…Scientific method is attunement to the world, not to ourselves.”(Adler, 1998, p. 44)

21. Scientific Method Approach to acquiring knowledge that involves formulating specific questions and then systematically finding answers 3 Principles of the Scientific Method Science is empirical Answers are obtained via structured observations Science is public Observations that scientists make are made available to others so that studies can be replicated and others can make use of the information Science is objective Studies are set up to minimize bias on the part of the researcher You wouldn’t want a researcher to find that a treatment works just because he wants it to!

22. As Behavior Analysts, we… Maintain a healthy skepticism Differentiate opinions, beliefs, and speculations from facts Don’t make claims without supporting objective data Skepticism ≠ cynicism Being open and being skeptical might seem contradictory, but they’re not. This is the way to think scientifically. Some phenomena that seem outlandish are valid Lucid dreaming, extraordinary feats of human memory and appropriate clinical uses of hypnosis (as opposed to the scientifically unsupported use of hypnosis for memory recovery). “We must keep our minds open but not so open that our brains fall out”

23. “Can’t you bend a little on the science?” Science provides certain criteria to decide what information is (and is not) valid (accurate) Actually, “validity” of information is along a continuum (from “baloney” all the way to “fact”) Ultimate goal for autism treatment: to identify what maximally benefits the child in need of intervention (and what doesn’t work or may even harm the child!) 23

24. “Those who fall in love with practice without science are like a sailor who enters a ship without a helm or compass, and who never can be certain whither he is going”-Leonardo da Vinci

25. Consumer of new research findings • Evaluator of your own interventions using empirical (scientific) methods • Researcher who produces and reports data to the applied and scientific community The Roles of the Scientist Practitioner

26. Gina Green “Evidence-Based Practice: Improvement or Illusion?” -ABAI Autism Conference 2008 Video:

27. Evidence-Based Practice

28. “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (APA Presidential Task Force on Evidence-Based Practice, 2006, p. 273) • http://www.apa.org/practice/ebp.html • Other similar terms • Empirically-validated therapies (EVT) • Empirically-supported therapies (EST) • Detrich (2008) argues that this term be reserved for interventions based on scientific knowledge Evidence-Based Practice (EBP)O’Donohue and Ferguson (2006)

29. 20 years ago - Clinical practice guidelines developed in medicine to • Help standardize decision-making in treatment • Encourage use of empirically sound treatments • Improve quality of medical services and reduce errors • Early 90s – American Psychiatric Association developed their own clinical practice guidelines • Intended to assist psychiatrists in decision-making in treatment • Tended to recommend pharmacotherapy over non-pharm treatments • Mid 90s – American Psychological Association (APA) Task Force publishes their own clinical practice guidelines… Development of EBPO’Donohue and Ferguson (2006)

30. Division 12 of the American Psychological Association (APA) – Clinical Psychology • Established a Task Force (headed by Diane Chambless) to identify and promote empirically supported psychological treatments in 1993. • Chambless et al. (1998) – available at http://www.apa.org/divisions/div12/est/97report.pdf “The Chambless Criteria”

33. Do not take clinical significance into account • Focus on efficacy, not effectiveness • Will the same results be obtained in natural settings? Is the treatment practical to implement in terms of staffing, funds, expertise? • Most studies on the EBP list do not include participants with comorbid disorders • Are biased toward group designs and inferential statistics • Only 2 “good” group design exps, but at least 9 single-case exps • Many single-case designs don’t compare 2 treatments Weaknesses of Chambless Criteria O’Donohue and Ferguson (2006)

34. Single-subject research documents a practice as evidence-based when… • The practice is operationally defined • The outcomes and context in which the practice is to be used are defined (target behaviors affected, setting, age, skills, diagnosis, implementer) • The practice is implemented with fidelity (tx integrity data) • The change is the DV is shown to result from the IV – experimental control! • Effects of the practice are replicated across a sufficient number of studies Horner, Carr, Halle, McGee, Odom, and Wolery (2005)

35. Scientific Merit Rating Scale • Scoring system to evaluate studies • 5 dimensions (each given score 0-5; max 25) • Research design • Measurement of DV • Measurement of IV (treatment integrity) • Participant ascertainment • Generalization of tx effects • Scoring of treatment effects • Beneficial • None • Adverse The National Standards Projecthttp://www.nationalautismcenter.org/about/national.php

36. Strength of Evidence Classification System • Criteria for receiving label of • Strongest support • Strong support • Modest support • Emerging treatment • Unestablished treatment • Discredited treatment The National Standards Projecthttp://www.nationalautismcenter.org/about/national.php

37. Measurement: to quantify the characteristics of what (or whom) you’re looking at (or to label it) • “Diagnosis” is a type of measurement involving labeling • “severity” of a disorder on a scale of 1-10 is a type of measurement involving quantification • “number of requests” made by a child in an hour is another measurement involving quantification Some Terms

38. Data Data: the collected measurements of what you’re examining These provide evidence about how good our information is In psychology, the word data is plural and should be used with plural verbs and modifiers. Incorrect: This data is interesting. Correct: These data are interesting.

39. Johnston and Pennypacker (1993) “As with other subcultures, science has its own traditions, rules, and manners. Some are quite important, and others are only signs of ‘good breeding.’ Properly using the singular and plural forms of the noun ‘datum’ probably falls into the latter category. ‘Datum’ properly refers to a single observation, whether in ‘raw’ or quantified form. When there is more than one such observation, these observations should be described with the word ‘data,’ which is the plural form of datum. Because “data” is a plural form of a noun, it requires the plural form of a verb, as in ‘The data are now ready for analysis.’ So, if you are conversing at the bar at the annual meeting of the ‘Nobel Prize Contenders in Waiting’ and should be so gauche as to say ‘data is…,’ you may later be quietly asked to resign. Should you slip and actually follow data with a singular verb while delivering your presidential address, you may actually cause heart failure in weaker members of the association. Don’t say you weren’t warned.” (p. 9)

40. Practice! The data suggest / suggests that the treatment was effective. The data for all participants is / are presented in Figure 1. A data path is / are created by connecting data points. These data indicate / indicates that DRO was more effective than response cost. The present data contribute / contributes to previous research. Data collection was / were conducted over a 5-week period. Multiple data paths is / are used in an alternating treatments design. The data is / are located on page 2. This data set demonstrate / demonstrates the effectiveness of this intervention. Data was / were collected in a laboratory at WMU. Data collectors is / are trained observers. This data path show / shows responding during the attention condition. Each data point on the graph represent / represents responding during a 15-min session. The data is / are displayed in Figure 1.

41. Variable: anything that can be measured and that can change in value (what you are investigating) • Dependent Variable: the part of the child’s behavior you are measuring (outcome) • Independent Variable: treatment Some Terms

42. Hypothesis: a statement predicting how one variable will be related to, or affect, another variable • It is not really an educated “guess” but a well informed prediction based on past evidence • Theory: a statement explaining why a relationship exists between two variables • What makes a theory a good one or a bad one? • Shall we talk about Bettelheim’s Refrigerator Mother theory? Some Terms

43. Operational Definition: an objective definition of a variable (everyone can measure what you are measuring) • Interobserver Agreement (IOA): extent to which observers agree on what they are observing • IOA will be high ONLY when the variables have been defined so that everyone understands what is being studied Some Terms

44. Functional Relationship: degree to which the independent variable affects the dependent variable (and can you isolate this effect!!) • Main focus in research in autism treatment is to determine functional relationship between treatment X and behavior Y • (Does the treatment benefit the child? AND Can we conclude that it was NOT something else that benefited the child?) Some Terms

45. Correlation: two variables seem to change in a regular fashion with one another • Big Problem: way too many people infer causality from this kind of relationship when we don’t really know, and they then act on it • AUTISM AND MMR VACCINES? Causation vs. Correlation

46. Causation: when evidence collected shows us that when one variable alone changes, the second variable also changes as a result of the first variable! • Using an experiment is what allows us to make such inferences of causality • Facilitated communication Causation vs. Correlation

47. To reiterate, goal is to determine functional relationships (for this class: how does a particular treatment affect a child?) • To do this, need to rule out effects of other variables you are not studying (called extraneous variables) • You control for the other extraneous variables by holding them as constants while varying the treatment (independent variable)… Experimental Designs

48. One way to do this is to observe a single person, or a group of persons, under one condition (“treatment”) and then another (“control condition”) without letting other variables change! Why? • Another way to do this is to get two or more groups of people who are the same, and then apply the treatment to one group (“treatment group”) but not the other (“control group”) without letting other variables change! Why? Experimental Designs

49. The more frequently we see a functional relationship demonstrated, the more sure we are that it is a valid relationship • Also tells us who the treatment applies to and who it doesn’t! Replication

50. We can be duped by others if we don’t know the research!!! • Peer-reviewed journals contain the most VALID research findings (beware of pseudo-scientific journals!) • Studies are written up in “APA style” which is an agreed upon way to present research findings (from American Psychological Association) Reading (and Writing) About Research Studies