1 / 28

Building a simple Poisson model in HLM: ln(FREQRND ij ) = π 0j + π 1j *(PHASE) + e ij

Learn how to construct a straightforward Poisson model in Hierarchical Linear Modeling (HLM) using ln(FREQRNDij) = π0j + π1j*(PHASE) + eij. This model helps analyze the behavior of children during different phases of a study.

rmoody
Download Presentation

Building a simple Poisson model in HLM: ln(FREQRND ij ) = π 0j + π 1j *(PHASE) + e ij

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building a simple Poisson model in HLM: ln(FREQRNDij) = π0j + π1j*(PHASE) + eij β 00 = average baseline frequency β10 = average phase effect Dicarlo & Reid (2004) *Note: In previous analyses, session slopes didn’t add anything significant to prediction, thus a simplified model is presented here.

  2. Dicarlo & Reid (2004) How does the average child behave? ln(acts) ≈ -0.80 + 2.56*(PHASE) ------------------------------------------------------------ Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ------------------------------------------------------------ For INTRCPT1, P0 B00 -0.800345 0.604807 -1.323 4 0.256 For PHASE slope, P1 B10 2.555173 0.289583 8.824 4 0.000 ----------------------------------------------------------- 2

  3. Dicarlo & Reid (2004) Interpreting HLM Fixed Effects Estimates • Nonlinear models: Not straightforward • Dependent variable may be hard-to-interpret function • Logarithm of a count, or ln(rate) = ln(count/base) • For “average” person: ln(actsij) ≈ -0.80 + 2.56*(PHASE) • For the baseline phase, the average value of ln(actsij) = -0.80 • Typical number of acts during a 10-minute session in the baseline phase is exp(-0.80) = 0.45 behaviors • Change from baseline to treatment phase, on average, is exp(2.56) = 12.94, i.e. 13-fold increase • The typical number of acts during a 10-minute session in the treatment phase is then 0.45*12.94 = 5.82 behaviors 3

  4. Dicarlo & Reid (2004)How much variation across children? ---------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ---------------------------------------------------------- INTRCPT1, R0 1.27 1.61814 4 27.14 0.000 PHASE slope, R1 0.45 0.19945 4 3.877 >.500 ---------------------------------------------------------- • Variance component for the estimation of baseline counts (intercepts) is “large” and statistically significant • I.e., student base levels vary (but all are still small) • Estimated variation in treatment effect is non-significant but large enough to bear watching (SD = 0.45). • Statistical significance ≠ practical importance

  5. Dicarlo & Reid (2004) Adding a Level-2 Predictor Allowing subjects’ cognitive functioning (expressed by cognitive age in months, minus 15) to explain between-subject variance in intercepts (baseline frequency): Level-1 Equation: ln(FREQRNDij) = π0j + π1j*PHASEij Level-2 Equations: π0j = β00 + β01*(COGAGE15j) + r0j π1j = β10 + r1j 5

  6. Dicarlo & Reid (2004) HLM Output from Poisson Model with COGAGE15 on Intercept: Final estimation of fixed effects: (Unit-specific model) --------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f P-value ---------------------------------------------------------- For INTRCPT1, P0 INTRCPT2, B00 0.041278 0.221231 0.187 3 0.864 COGAGE15, B01 -0.300262 0.040707 -7.376 3 0.000 For PHASE slope, P1 INTRCPT2, B10 2.397366 0.203890 11.758 4 0.000 ---------------------------------------------------------- ln(Acts) ≈ 0.04 - 0.30 * COGAGE15 + 2.40 * PHASE 6

  7. Dicarlo & Reid (2004)Interpreting HLM Estimates ln(Acts) ≈ 0.04 - 0.30 * COGAGE15 + 2. 40 * PHASE • For a typical student close to the sample’s median age of cognitive functioning (COGAGE15=0), the expected count of play actions per session during baseline is exp(0.04) = 1.04 • For every 1-month increase in cognitive functioning, a student’s estimated log count of pretend play actions per session during the baseline phase is estimated to change by -0.30 • For a student with a cognitive age of 16 months (COGAGE15=1), the expected count of play actions per session during baseline is 1.04*exp(-0.3003) = 1.04*0.74 = .77 • For a student with a cognitive age of 14 months (COGAGE15=-1), the expected count of play actions per session during baseline is 1.04*exp(0.3003) = 1.04*1.35 = 1.40

  8. Dicarlo & Reid (2004)Interpreting HLM Estimates (cont’d) ln(Acts) ≈ 0.04 - 0.30 * COGAGE15 + 2. 40 * PHASE • Number of acts in treatment phase is expected to be exp(2.3974) = 10.99 times as large as during baseline • The typical number of acts during a 10-minute session in the treatment phase, for a student with a cognitive age at about 15 months (COGAGE15=0) is then 1.04*10.99 = 11.43 • For a student with a cognitive age of 16 months,0.77*10.99 = 8.46 acts are expected per session in the treatment phase • For a student with a cognitive age of 14 months, 1.40*10.99 = 15.39 acts are expected per session in the treatment phase

  9. Dicarlo & Reid (2004) Test for remaining variation among children Final estimation of variance components: ---------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ---------------------------------------------------------- INTRCPT1,U0 0.18357 0.03370 3 3.79574 0.284 PHASE slope,U1 0.01056 0.00011 4 2.34291 >.500 ---------------------------------------------------------------------------------------------------------- Residual variances for intercept and phase effect are now small and non-significant (though power is low). 9

  10. Dicarlo & Reid (2004) Exploring the possibility of an outlier subject One subject (Kirk) has no observed pretend play actions during any session in the baseline phase: He also happens to have the highest measured cognitive functioning of all subjects in the sample (which we just found to significantly predict baseline counts). So, is it cognitive functioning or just being “Kirk” that matters for baseline estimates? 10

  11. Dicarlo & Reid (2004) Exploring the possibility of an outlier subject HLM Estimates of Random Effects: Final estimation of variance components: ----------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ----------------------------------------------------------- INTRCPT1,R0 0.51505 0.26527 3 11.10273 0.011 PHASE slope,R1 0.22279 0.04963 4 2.18264 >.500 ----------------------------------------------------------- 11

  12. Residual variances for intercept and phase effect are small • But not as small as those from the model that included COGAGE15 instead of KIRK. • Cognitive functioning better explains variation in baseline counts than does knowing whether or not a subject is Kirk. • Nonetheless, I prefer the model with Kirk as outlier: • Predictions for model with age are odd • Effect of each year of cog. age is too large to believe • Therefore, it’s being distorted to fit Kirk’s behavior • Sometimes substantive info trumps statistical fit

  13. Example: Nonlinear Model • Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165-179. • Three respondents, with phase changes at different points in time

  14. Horner et al Fig 2: Initial Observations • Each point on the graph is the proportion of correct trials (presumably out of 20) each session • Reliability of estimate of p(correct|session) depends on number of trials (usually ignored in visual analysis) • Each plot starts low (at or near zero), then climbs, and flattens at/near probability one (100 percent)

  15. Horner et al Fig 2, Cont. • Form of each curve is ogive; relatively smooth curve, often found in analysis of categorical data (logistic regression) • Curves are displaced as a function of when phase changed • No obvious immediate jump when phase changes

  16. At what point on X does the probability reach .5?How does this vary if the curve is moved to the left or right?

  17. LD50 (equiv: Item difficulty in IRT) • Simple logistic model: • ln(odds) = b0 + b1 * X • When does p = .5 ? • odds = .5 /.5 = 1 • logit = ln(odds) = ln(1) = 0 • If 0 = b0 + b1 * X, then • X = -b0 / b1 • In SCD, each person may have a different value of b0 and b1

  18. Initial Research Questions • Do curves have same slope for each person? (We will assume “yes” here, but can test it.) • Do curves have same “LD50” or “b”? (Point of 50 percent response) • Is there a jump at phase change? If so, same for each participant?

  19. Fixed Effects Estimates (Logistic Model)(Note HLM will not allow direct estimation of LD50 ) Final estimation of fixed effects: (Unit-specific model) --------------------------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value --------------------------------------------------------------------------------------------- For INTRCPT1, B0 G00 -0.85 0.356 -2.381 2 0.093 For PHASE slope, B1 G10 0.72 0.241 2.994 58 0.004 For SESS.10 slope, B2 G20 0.14 0.022 6.508 58 0.000 ---------------------------------------------------------------------------------------------

  20. Fitted Model for Person 1 • Lower line is prediction for baseline phase • Upper line is prediction for treatment phase • Vertical line is phase shift (between sessions 6 and 7)

  21. Variance estimate and its implication ------------------------------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ------------------------------------------------------------------------------------------------- INTRCPT1, U0 0.56602 0.32037 2 50.12025 0.000 ------------------------------------------------------------------------------------------------- • Standard deviation of intercepts is significantly greater than 0 (even with only 3 people, and 2 df) • The respondents intercepts differ. • In this case, where slopes are equal, this is equivalent to “b” or “LD50” (displacement to right or left) differing across respondents • Two strands of evidence point to effectiveness of treatment • Phase effect indicates a jump at shift to treatment phase • Significant variance in intercepts indicates shift in point where curves rise

  22. Nuances to Add to Model • Baseline may not be at 0; requires logistic with floor effect • Upper asymptote may not be 1; requires ceiling effect • Any of these (slope, LD50, jump, floor, ceiling) may vary across people • Fixed (may be needed with small N) • Random (may be ok with larger N)

  23. Summary: Simple Models Aren’t So Simple • Difficulty fitting quadratic random effects • Difficulty estimating AR model • AR results sensitive to misspecification • Poisson, binomial (logistic) models • Harder to interpret/explain • Problems with many zeros 27

  24. Acknowledgments • Will Shadish for being a perfect PI-partner • Eden Nagler for her hard work, and her organizational and analytical skills, without which I wouldn’t be past the first data set by now • NIE/IES for financial support, Grant H324U050001, Methods for Meta-Analysis of Single-Subject Designs

More Related