Analyzing Data from Small N Designs using Multilevel Models

1. Analyzing Data from Small N Designs using Multilevel Models Eden Nagler The Graduate Center, CUNY David Rindskopf, Ph.D The Graduate Center, CUNY

2. Overview/Intro • What is our current work? • Where did we start? • How does HLM fit into this framework?

3. 2 Initial Datasets: Stuart, R.B. (1967). Behavioral control of overeating. Behavior Research & Therapy, 5, (357-365). Dicarlo, C.F. & Reid, D.H. (2004). Increasing pretend toy play of toddlers with disabilities in an inclusive setting. Journal of Applied Behavior Analysis, 37(2), (197-207).

4. Stuart (1967):

5. Stuart (1967):Procedures for Getting data into HLM

6. Stuart (1967): Procedures for Getting data into HLM

7. Stuart (1967): Level-1 dataset

8. Stuart (1967): Level-2 dataset

9. Stuart (1967): HLM (Linear model) Linear Model: POUNDS = π0 + π1*(MONTHS12) + e

10. Stuart (1967): HLM – Linear Model Estimates Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------- For INTRCPT1,P0 INTRCPT2, B00 156.439560 5.053645 30.956 7 0.000 For MONTHS12 slope, P1 INTRCPT2, B10 -3.078984 0.233772 13.171 7 0.000 ---------------------------------------------------------- The outcome variable is POUNDS ---------------------------------------------------------- POUNDSij ≈ 156.4 – 3.1*(MONTHS12) + eij

11. Stuart (1967): HLM – Quadratic Model Quadratic Model: POUNDS = π0+ π1*(MONTHS12)+ π2*(MON12SQ)+e

12. Stuart (1967): HLM – Quadratic Model Estimates Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ----------------------------------------------------------- For INTRCPT1, P0 INTRCPT2, B00 158.833791 5.321806 29.846 7 0.000 For MONTHS12 slope, P1 INTRCPT2, B10 -1.773039 0.358651 -4.944 7 0.001 For MON12SQ slope, P2 INTRCPT2, B20 0.108829 0.021467 5.070 7 0.001 ----------------------------------------------------------- The outcome variable is POUNDS ----------------------------------------------------------- POUNDSij ≈ 158.8 – 1.8(MONTHS12) + 0.1*(MON12SQ) + eij

13. Stuart (1967): HLM – Linear vs. Quadratic Model Stuart (1967) – Actual Data Linear Model Prediction Quadratic Model Prediction

14. Dicarlo & Reid (2004):

15. Dicarlo & Reid (2004): Level-1 dataset

16. Dicarlo & Reid (2004): Level-2 dataset

17. Dicarlo & Reid (2004): HLM – Simple Model Simple Model: FREQRND = π 0 + π1*(PHASE) + e

18. Dicarlo & Reid (2004): HLM – Simple Model Estimates Level-1 Model Level-2 Model log[L] = P0 + P1*(PHASE) P0 = B00 + R0 P1 = B10 + R1 ---------------------------------------------------------- Final estimation of fixed effects: (Unit-specific model) Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------- For INTRCPT1,P0 INTRCPT2, B00 -0.769384 0.634548 -1.212 4 0.292 For PHASE slope,P1 INTRCPT2, B10 2.516446 0.278095 9.049 4 0.000 ---------------------------------------------------------- LN(FREQRNDij) = -0.77 + 2.52*(PHASE) + eij

19. Dicarlo & Reid (2004): HLM – Simple Model Estimates LOG(FREQRNDij) = B00 + B10*(PHASE) + eij For PHASE=0 (BASELINE): LOG(FREQRNDij) = B00 FREQRNDij= exp(B00) For PHASE=1 (TREATMENT): LOG(FREQRNDij) = B00 + B10 FREQRNDij= exp(B00+B10) = exp(B00)*exp(B10) Estimates: B00 = -0.77; B10 = 2.52 For PHASE=0 (BASELINE): FREQRNDij= exp(B00) = exp(-0.77) = 0.46 For PHASE=1 (TREATMENT): FREQRNDij= exp(B00+B10) = exp(-0.77+2.52) = exp(1.75) = 5.75

20. In conclusion… • Other issues we’ve encountered and explored • Issues we’ve encountered, but not yet explored • Issues we’ve not yet encountered nor explored