A d-Estimator For Single Case Designs

1. A d-Estimator For Single Case Designs Larry V. Hedges Northwestern University Presented at the Fifth Annual IES Research Conference National Harbor, MD, June 2010

2. Important Note This work is part of a joint project with: Will Shadish and David Rindskopf But they didn’t get a preview of this presentation and aren’t here to defend themselves, so don’t blame them …

3. Goals Describe a perspective that could lead to a d-estimator for single subject designs Sketch a program of work that follows that perspective Show some simple results Describe where we plan to go from here

4. Perspective Effect size measures are useful for representation of results of single studies Effect size measures are essential for cumulation of results across studies in the ways that are becoming conventional (e.g., WWC, meta-analysis) Accumulation of results is essential to make research count in some scientific and policy contexts An effect size measure for single subject designs that was comparable to those used in between-subjects designs would be desirable

5. What Does it Mean to Be Comparable? Imagine a research design with both time series data and between-subjects data A subset of the total data could be either • A single subject design • A between subjects design The effect size (d-index) from the between subjects design is defined We want to create an effect size for the single subject design that estimates the same parameter as the d-index for the between subjects design In principle we could estimate the same effect size from either design

6. A two period study with stationary time series structure Time series structure Baseline Yij = μi + εij, j = 1, …, ni1εij ~ N(0, σ2) Treatment Yij = μi + δi + εij, j = ni1 + 1, …, ni1 + ni2εij ~ N(0, σ2) Between-subjects structure μi~ N(μ●, τ2) , δi ~ N(δ●, θ2), i = 1, …, m Effect Size Parameter

7. Where Does d Come From? The variance (across subjects) of any observation (any Yijacross i) at baseline is τ2 + σ2 The mean difference between treatment and baseline is δ● Thus is the d-index in the between subjects design that randomly assigned subjects to baseline or control and took one observation So all we need to do is figure out how to estimate d from a single subject design

8. Autocorrelation Structure The εij have an autocorrelation structure across j

9. How Could We Estimate d in a Single Subject Design? To estimate d we need to estimate δ●,τ2, and σ2 estimatesδ● Estimation of σ2 is complicated by the autocorrelation structure of the εij’s Estimation of τ2 and is complicated by the fact that the variance across people includes σ2 But both of these just require a little more work using relatively standard ideas One major caveat is that we can only estimate τ2 if there are replications within the study, but this is common in single subject research

10. How Could We Estimate d? Assuming a first order autocorrelation of ρ, and phases of length n, the estimate of σ2 is The estimate of τ2 is Thus the effect size can estimated from single subject results .

11. How Could We Estimate d? Putting all of this together we get with variance

12. Important Caveats These results assume: • Stationarity (differencing may be necessary) • First order autocorrelation only • Known value of first order autocorrelation • Replications within a study (to get information to estimate τ2) • Normality We are currently looking into empirical values of autocorrelations—they appear to be small on the average

13. Future Directions Extending methods to several treatment and baseline periods Development of empirical information on ρ, n, and m Checking small sample accuracy of approximations to the distribution of d Checking robustness of sampling distributions to violations of assumptions about ρ and the assumption of a first order autoregressive model Development of software to implement these methods (formulas get a little tedious with unequal n’s)

14. Thank You!