Current Methodological Issues in Single Case Research

1. Current Methodological Issues in Single Case Research David Rindskopf, City University of New York Rob Horner, University of Oregon

2. Purposes • Inclusion of Single-case Designs in education research literature. • Interpretation of Single-case Research • Visual Analysis • Statistical Analysis • Impact of Single-case Research • Identification of “Evidence-based Practices” • Inclusion of Single-case Research in Meta-analyses

3. Toward a Comprehensive Analysis of Single-case Designs Rob Horner, George Sugai, Hariharon Swaminathan, Tom Kratochwill, and Joel Levin

4. Focusing the Discussion • We are concerned here only with single-case experimental designs* • ABAB Reversal/withdrawal Designs • Multiple Baseline Designs • Alternating Treatment Designs • Note: these designs are sometimes conceptualized as within, combined, and between-series, respectively. • Excluded are AB designs, ABA designs, Interrupted Time Series Case Studies. • The three types of designs can be extended into randomized experiments (Kratochwill & Levin, 2007).

5. Single Case Research Logic • Individual is unit of analysis • Dependent Variable • Operationally defined • Measured directly • Repeated measurement • Independent Variable • Measured for implementation fidelity • Replication of effect within an experimental design • To control for threats to internal validity

6. Analysis Needs X X X

7. Visual Analysis • Documenting Experimental Control • Three demonstrations of an “effect” at three different points in time. • An “effect” is a predicted change in the dependent variable when the independent variable is actively manipulated. • Visual Analysis includes simultaneous assessment of: • Level, Trend, Variability, Immediacy of Effect, Overlap across Adjacent Phases, Similarity of Data Pattern in Similar Phases. • (Parsonson & Baer, 1978; Kratochwill & Levin, 1992)

8. Visual Analysis • Interpreting experimental control always involves assessment of data from the whole study(all phases), not just assessment of two adjacent phases. • Assessment of an “effect” is done with 2 adjacent phases. • Assessment of experimental control, however, requires evaluation of all data in all phases.

9. First Demonstration of Effect Third Demonstration of Effect Second Demonstration of Effect Visual Analysis: 1. Change in Level 2. Change in Trend 3. Change in Variability 4. Immediacy of Effect 5. Overlap 6. Similarity of Data Patterns and Phases Parsonson & Baer, 1978; Kratochwill & Levin, 1992

10. First Demonstration of Effect Third Demonstration of Effect Second Demonstration of Effect Comparison of actual against projected data (Analysis of Transition States versus Analysis of Steady States)

11. First Demonstration of Effect Third Demonstration of Effect Second Demonstration of Effect

12. First Demonstration of Effect Second Demonstration of Effect Third Demonstration of Effect

14. Statistical Analyses of Single-case Research • Documenting Experimental Control/ Statistical Significance: • No statistical model currently replicates the conceptual logic used in visual analysis. • Effect Size • Not a variable traditionally addressed in Single-case Analyses. • It needs to be.

15. Documenting “Evidence-based Practices” via Single-case Research • Define “practice” • Build professional consensus

16. Defining a “Practice” • A “practice” is a procedure, or set of procedures, designed for use in a specific context, by individuals with certain skills/features, to produce specific outcomes for specific individuals. • Operationally defined procedures • Target population/ Context • Implementer Characteristics • Defined outcomes • Large practices…small practices

17. Toward a Professional Consensus • Five studies documenting experimental control • Conducted in at least three different locations by at least three different researchers • At least 20 different participants • Documenting an effect size of ??

18. Example of using Single-Case Research to document Evidence-based Practice • Functional Communication Training to reduce problem behavior in school, home and community • At least Five Studies • Bird, Dores, Moniz & Robinson (1989) • Brown et al., (2000) • Carr & Durand (1985) • Durand & Carr (1987) • Hagopian, Fisher, Sullivan, Acquisto & LeBlanc (1998) • Mildon, Moore, & Dixon (2004) • Wacker et al., (1990) • At least three settings /scholars • Six locations, Research Groups • At least 20 participants • 42 different participants document effect

19. Implications for Single-Case Research • Measures • Define assumptions about trajectory • Baseline • Document predictable patterns • More data points if trend • More data points if higher variability • Data points per phase • At least 5 points per phase • More data when trend and variability is high • Combination of visual and statistical analysis • Need for effect size measures in single-case designs • For individual studies • For meta-analyses

21. Summary • The inclusion of single-case research findings in a larger body of education research will require attention to added strategies for interpretation, and added strategies for application • Interpretation • Better definition of visual analysis procedures • Agreement on effect size index • Application • Definition of “evidence-based practices” • Protocol for inclusion of single-case research in meta-analyses.