1 / 23

NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI

NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI . Jill Pentimonti Adrea Truckenmiller Jessica Logan Sara Hart Discussion: Grandpa Schatschneider Presented Feb 6, 2014 Pacific Coast Research Conference, San Diego.

alamea
Download Presentation

NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI Jill Pentimonti Adrea Truckenmiller Jessica Logan Sara Hart Discussion: Grandpa Schatschneider Presented Feb 6, 2014 Pacific Coast Research Conference, San Diego

  2. Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart & Grandpa Florida State University

  3. Expanding our search for moderators of intervention • A little about me • Behavioral genetics background • PCRC participant • Even with modest effect sizes, individual differences in intervention response • Bioecological model (Bronfenbrenner & Ceci, 1994) • Provides framework for differentiating students based on non-intervention related traits

  4. Integrative Data Analysis (IDA) • Item-level pooled data (Curran & Hussong, 2009) • Capitalizes on cumulative knowledge • Longer developmental time span • Increased statistical power • Increased absolute numbers in tails • Controls for heterogeneity • Sampling, age/grade, cohort, geographical, design, measurement

  5. Project KIDS • Expanded definition of moderators of response to intervention • Cognitive, psychosocial, environmental, genetic risk • IDA across 9 completed intervention projects • Approximately 5600 kids • Data entry of item level data common across at least 2 projects • ~30 different assessments • Questionnaire data collection

  6. Proof of Concept • Behavior problems and achievement are associated • More behavior problems are typically seen in LD populations • Is adequate vs inadequate response status differentiated by behavior problems?

  7. Method • Participants • 2005-2006 ISI intervention project (Connor et al., 2007) • RCTish : 22 treatment, 25 contrast teachers, 3 pilot • 821 first graders • A2i recommendations vs standard practice • 2007-2008 ISI intervention through FL LDRC (Al Otaiba et al., 2011) • RCT: 23 treatment, 21 contrast teachers • 556 kindergarteners • A2i recommendations vs enhanced standard practice

  8. Method • Measures • WJ Tests of Achievement Letter-Word Identification (LWID) • Pre- and post-intervention testing periods • Social Skills Rating Scale: Behavioral Problems subscale • Teacher completed during intervention year

  9. Results: Calibration LWID • Randomly selected 1 time point/child/project to form “calibration sample” for LWID • IRT with decision to include only items > 5% endorsement rate • Reduced item sample from 75  36 • Items 8 to 44

  10. Results: Calibration LWID • Generalized linear factor analysis (GLFA) • Combines latent factor analysis and 2-PL IRT model • Here, equivalent of 2-PL IRT model with DIF • No significant DIF was found

  11. Results: Second data sample LWID • Using remaining data, GLFA model run again, setting parameters based on calibration sample • Separately by project • If significant, add DIF estimates to parameters

  12. Results: SSRS • IRT to GLFA model with Project DIF on full data

  13. Results

  14. Results: Response • Proc mixed: covariance adjusted LWID score • 1169 children

  15. Results: Response • 648 treatment children

  16. Results: Response • 648 treatment children Unresponsive Cutoff < 20% N=110!

  17. Results: Response • 648 treatment children Unresponsive Cut off Fall SS = 95 Spring SS= 104 Mean Fall SS = 86 Spring SS = 96

  18. Results: Response • 648 treatment children Responsive Mean Fall SS = 99 Spring SS = 111 Unresponsive Cut off Fall SS = 95 Spring SS= 104 Mean Fall SS = 86 Spring SS = 96

  19. Results • Logistic regression • SSRS behavior problems significant predictor of response status (OR = 1.45, CI = 1.12-1.88) • average behavior problems = 19% probability of being “unresponsive” • greater than average behavior problems(+ 1SD) = 29%probability of being “unresponsive” • Less than average behavior problems (-1SD) = 12% probability of being “unresponsive”

  20. Conclusions • Response status is differentiated by behavior problems • Mo’ behavior problems, mo’ (reading) problems! • The questionnaire data we will be adding will be real test of bioecological model on response to intervention

  21. Overall IDA conclusions • IDA is a “cheap” way to get more power, more n at tails, and show more generalizable effects • Given how similar many of our projects are, consider doing item-level data entry • Easy potential to combine data • Can you do factor analysis and IRT? You can do IDA! • These data are more useful together than apart • IRT within and between samples? • Treatment effectiveness across samples? • Characteristics of lowest responders?

  22. Acknowledgements • Stephanie Al Otaiba • Carol Connor • Chris Schatschneider • Great staff & grad students, and a small army of data enterers NICHD grant HD072286

More Related