1 / 29

An Incomplete Model for Trial Designs for Moving to Scale: The Whole Day Program

An Incomplete Model for Trial Designs for Moving to Scale: The Whole Day Program. C Hendricks Brown Sheppard Kellam Jeanne Poduska Amy Windham John Reid Carla Ford Natalie Keegan Supported by R01 DA15409-02 PI: Kellam Ro1 MH40859-17 PI: Brown Baltimore City Public School System.

sema
Download Presentation

An Incomplete Model for Trial Designs for Moving to Scale: The Whole Day Program

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Incomplete Model for Trial Designs for Moving to Scale: The Whole Day Program C Hendricks Brown Sheppard Kellam Jeanne Poduska Amy Windham John Reid Carla Ford Natalie Keegan Supported by R01 DA15409-02 PI: Kellam Ro1 MH40859-17 PI: Brown Baltimore City Public School System Hopkins PSMG Oct 2005

  2. Premise The Whole Day Program is where the school would want to be in 5 years, systemwide. What specific steps should be examined along the way? Hopkins PSMG Oct 2005

  3. Primary Question Effective? Who benefits, for how long? Who doesn’t benefit, who is harmed? Efficacy Trial 12 Schools 24 Classrooms 600 Students 3 Time Periods Hopkins PSMG Oct 2005

  4. Differential impact on spring of first grade reading achievement by baseline reading (fall of first grade): control against two interventions Classroom / Parent Interventions Control Hopkins PSMG Oct 2005 Brown: Statistical Power

  5. Presuming Effectiveness Sustainabililty Do WD teachers continue to sustain the same effects on children that they have in following years? 12 WD Teachers vs Themselves & 12 Wait-Listed Controls Scalability Can program be successfully implemented outside the original 12 schools throughout the school system? Hopkins PSMG Oct 2005

  6. Scale to What? • From 2 To all 4 of the School Areas • From 12 To 20 To 135 Schools • From 24 To 40 To 400 First-Grade Classrooms Hopkins PSMG Oct 2005

  7. An Evidence-Based Decision Tree Efficacious especially w/ High-Risk Youth? N Y Sustainable? Y N Feasible to Scale? Y N Go to Scale! Hopkins PSMG Oct 2005

  8. The Real Decision Tree for BCPSS Something Different Hopkins PSMG Oct 2005

  9. Person, Place, and Time • Child • Classroom / Teacher (1st Grade) • School • Area (Region of Baltimore) • Time – Scale(s): 10 Secs, Min, Quarter, Year • Child : Track from 1st through 3rd Grade • Teacher: Track over 3 years (cohorts) Hopkins PSMG Oct 2005

  10. Randomization • 20 Schools in 2 Areas randomly assigned to One of 12 schools where trial begins One of 8 schools that are “wait-listed” • Children randomly assigned to classes within school • Teachers/Classes randomly assigned to WD or wait-list Hopkins PSMG Oct 2005

  11. Baltimore Whole Day TrialAssignment of Classrooms/Teachers in First YearHalf the Teachers are Wait-Listed for WD to 3rd Year Replicate Same Design w/ SC and and WD BOTH in 12 Different Schools Hopkins PSMG Oct 2005 Brown: Statistical Power

  12. Effectiveness Evaluation Compare 3 years of Outcomes for Children in the 12 WD classes to those in 12 Standard classes -- 1st Cohort Examine Level of Program Implementation in the 12 WD and 12 Standard Classrooms in 1st Cohort Hopkins PSMG Oct 2005

  13. Classroom Observations by Independent Observers 3 times per year WD and SC classrooms AM-Reading Instruction PM-Other subjects Teacher Practices Timed Observations Checklist Global Ratings of Quality Student Behavior On-Task/Off-task Ratings of Disruptive, Aggressive, and Shy Behavior Hopkins PSMG Oct 2005

  14. Classroom Observation Data Collection FormTeacher Practices and Student Behavior Actual Form Hopkins PSMG Oct 2005

  15. Domains of Reading Instruction Word Work – phonemic awareness, alphabetic instruction, letter recognition, work work Reading Comprehension – reading comprehension, previewing a book, reading books Language – oral language, vocabulary, reading own writing, writing, grammar Non-reading Instruction – interruptions, teacher out of classroom Giving instructionally related reading directions Feedback in response to behavior Feedback in response to a reading relevant comment or event Hopkins PSMG Oct 2005

  16. Classroom Observations BaselineData Overview 12 schools 24 teachers (12 Whole Day, 12 Standard setting) Each teacher observed 3 times per year, at each time point: • 1 reading instruction (am observation) • 1 non-reading instruction (pm observation) Length of observations at each time point: Teacher: mean = 164 minutes Each student: mean = 9 min/student Hopkins PSMG Oct 2005

  17. Given Teacher Instructional Practices, Classroom Behavior(Standard Practice Classrooms during Reading Instruction at Baseline)Well behaved v Fairly well behaved: X2=36.83 p<10-7 Hopkins PSMG Oct 2005

  18. Student behavior: Reading v Other subjects(Standard Practice Classrooms at Baseline)Never off-task v Ever off-task: X2=29.80 p<10-8 Hopkins PSMG Oct 2005

  19. Effectiveness Results for Cohort 1 (1 of 2) • At Baseline Level of Off-Task Behavior is Well Balanced across Intervention Condition • Reduction in Off-task Behavior Control 21% WD 9% • 50% Reduction in Aggressive Behavior for Males in Less Structured Settings Hopkins PSMG Oct 2005

  20. Graphical Comparison for Off-Task Behavior Hopkins PSMG Oct 2005

  21. Comparison of Teachers Hopkins PSMG Oct 2005

  22. Unsolved Multilevel Modeling Issues Child Gender, Baseline Off-Task Classroom Intervention, Classroom Baseline Aggr School Time-Varying Classroom Context Binary Outcomes Hopkins PSMG Oct 2005

  23. Sustainability Evaluation (Funded) Multilevel Support Compare Teacher Implementation (and Child Outcomes) for 12 WD Teachers in 1st through 3rdYear for First Grade Classes Hopkins PSMG Oct 2005

  24. Sustainability Hypotheses and N’s & Years • Teacher practices over time will be related to the quality of WD integration at the level of the School Building Team. 12  20  135 2 • Teacher practices over time will be related to the quality of WD integration at the level of the School Area over time. 2  4 Hopkins PSMG Oct 2005

  25. Scalability Design Change in fidelity (Teacher practices) and impact (Child Outcomes) in succeeding years after initial training 12  24  135 Hopkins PSMG Oct 2005

  26. Combined RCT and Implementation TrialTrial After a Trial Design Hopkins PSMG Oct 2005

  27. How to Go From 24 to 400 Classes? A Dynamic Wait-Listed type of design Randomly Assign WHEN schools and classes obtain training, coaching, and full implementation Continue to assess intervention fidelity Paired-down measure of Teacher Practices Random Assign Children to Classes? Hopkins PSMG Oct 2005

  28. Classes Schools Waiting Trained Sustained Active School Wait Listed Trained Sustained Hopkins PSMG Oct 2005

  29. Plausible Time Scale # Classes (Schools) Hopkins PSMG Oct 2005

More Related