Effective Strategies for Building a Base in Education

1. Whole Day First Grade Program: Effectiveness, Sustaining, & Building a Base for Going to ScaleJeanne Poduska, ScDCenter for Integrating Education and Prevention Research in Schools, American Institutes for ResearchPresented at: New Directions for Mental Health and Drug Abuse Effectiveness and Dissemination Trial Research and MethodologyOctober 28, 2005

2. Contributors and Support Sheppard G. Kellam, M.D. 1 Jeanne Poduska, Sc.D. 1 C. Hendricks Brown, Ph.D. 2 Carla Ford, Ph.D. 1 Amy Windham, Ph.D. 1 Natalie L. Keegan 1 John Reid, Ph.D. 3 Bonnie Copeland, Ph.D. 4 Linda Chinnia, 4 National Institute on Drug Abuse, Division of Epidemiology, Services and Prevention Research, Prevention Research Branch 1R01 DA015409 and 1R01 DA019984 (and support from NICHD) 1 American Institutes for Research, Center for Integrating Education and Prevention Research in Schools 2 University of South Florida, Prevention Science Methodology Group 3 Oregon Social Learning Center 4 Baltimore City Public School System

3. Phases of Education and Prevention Trials

4. Developmental Epidemiology: A Strategy for Prevention Research Community Epidemiology Life Course Development Preventive Interventions

5. Research roots of WD • First Generation of Baltimore trials tested separately Mastery Learning (ML) aimed at reading, and Good Behavior Game (GBG) aimed at classroom management of student behavior in First Grade. • Each had short-term impact on its target; GBG had long term impact on very high risk children. • Second Generation tested combination of instructional improvement and GBG and showed impact on reading and behavior into middle school, still room for improvement. • A family/classroom program also seemed promising. • Oregon Prevention Research Center tested family/ classroom partnership (LIFT program) and found impact among higher risk children.

6. Add Outcome Slide

7. How do youEstablish Community & Institutional Partnerships? • Analyze social/political structure • Learn the vision of problems and priorities • Identify mutual self-interest within and across leadership • Fit your interest under the vision of leadership • Request ad-hoc oversight committee of leaders • Work through trust • This process takes about a year

8. Functions of a • Community & Institutional Board • Establish vision and priorities • Examine/critique/approve/support proposed programs • Communicate constituents’ concerns, values, priorities, and acceptable language to program leaders • Communicate program needs to constituent leaders • Continually assess absent constituencies • Work toward institutionalizing programs

9. Baltimore City Public School System Partners Patricia Welch, Ph.D., Chair, BCPSS Board of School Commissioners Bonnie Copeland, Ph.D., Chief Executive Officer Linda Chinnia, Chief Academic Officer Ben Feldman, Ph.D., Research, Evaluation & Accountability Officer Sue Cutter, Area 1 Interim Academic Officer James Smith, Area 2 Academic Officer Gwen Cleage, Curriculum and Instruction Officer La Verne Sykes, Director, Parent and Community Involvement Deborah Wortham, Ph.D., Director of Professional Development Delores Alexander, Coordinator of Reading Marietta English, President, Baltimore Teachers Union James Gittings, Public School Supervisors and Administrators Association 20 Schools, Principals, and Teachers Areas 1 & 2

10. Three Foci of Baltimore Whole Day First Grade Program (WD) • Improve reading instruction in all academic subjects thru the day, based on prior Baltimore trials. • Improve classroom behavior management, based on prior Baltimore trials. • Improve parent/classroom partnership re reading and behavior, based on prior Baltimore and Eugene, Oregon trials.

11. Lessons Learned From First Two Generations of Trials in Baltimore Without a system to mentor, model, and monitor teacher practices over time, practices are not sustained. A multi-level institutional structure with mentoring and monitoring is required to implement interventions with fidelity. Teachers need support from principals; principals from area leaders; area leaders from chief academic and executive officers.

12. Goals of WD Trial • Can we find synergy from the combination? • Can the practices be sustained over time? • Can practices be scaled up by school system with same effect?

13. Baltimore Whole Day First Grade Program Analytic Model Decreased Later Substance Abuse Decreased Aggressive, Disruptive Classrooms Decreased Individual Aggression Classroom Behavior Management Individual Classrooms Aggression Decreased Later Conduct & Anti- social Personality Disorders Teachers’ Effective Academic Instruction Increased Achievement Poor Poor Improved Reading Skills Reading Skills Achievement School Success & Decreased Drop-Out Decreased Depressive Symptoms Effective Family-Classroom Partnerships Depressive Symptoms Decreased Later Depressive Disorders Other mediating or moderating variables: • Family structure and poverty Whole Day First Grade Education and Prevention Program • Deviant peers Whole Day • • School building • • Community economic health, • resources, drugs, and violence •

14. Multi-Level Structure to Sustain Practices and Go to Scale BALTIMORE CITY PUBLIC SCHOOL SYSTEM Board of School Commissioners Chief Executive Officer Research and Evaluation Office Chief Academic Officer Professional Development Officer Area Academic Officers Lead Coaches Curriculum & Instruction Officer Principals Coaches Teachers

15. Core Team BALTIMORE CITY PUBLIC SCHOOL SYSTEM Board of School Commissioner Chief Executive Officer Research and Evaluation Office Chief Academic Officer Professional Development Officer Area Academic Officers Lead Coaches Curriculum & Instruction Officer Principals Coaches Teachers

16. Schools Committee BALTIMORE CITY PUBLIC SCHOOL SYSTEM Board of School Commissioner Chief Executive Officer Research and Evaluation Office Chief Academic Officer Professional Development Officer Area Academic Officers Lead Coaches Curriculum & Instruction Officer Principals Coaches Teachers

17. Random Teachers randomly assigned to classrooms SC (a) SC (b) WD Children randomly assigned to classrooms Random Within-school Random Assignment* of Teachers/Classrooms and Children in the Whole Day First Grade Program IN EACH SCHOOL: *Brown & Liao. AJCP, 1999.

18. Effectiveness Design of the Whole Day First Grade Program: Follow 2 Cohorts of 1st Graders to 3rd Grade

19. Sustainability Design of the Whole Day First Grade Program: Follow Teachers over Cohorts of 1st Graders

20. Scaling-Up Design of the Whole Day First Grade Program: 3 Cohorts of 1st Graders

21. Design of the Whole Day First Grade Program

22. Primary Measurement Objectives in the WD Trial • Implementation • Proximal effects of intervention on student behavior, achievement, and psychological well-being • Longer term (distal) effects • Teacher practices • Hypothesized or known mediating and moderating variables • Measures of the multi-level structure

23. Fidelity of Whole Day First Grade Program Implementation Checklist Excerpt

24. Measures of EffectivenessSocial Adaptational Status

25. Measures of EffectivenessPsychological Well-being

26. Teacher Practices Observations by Independent Observers (3x per year) • Teacher Practices • Timed Observation of Instruction (Foorman) • Global ratings

27. Measures of Multi-Level System to Support Sustaining Practices and Scaling-Up • Defined expectations for each level of the multi-level structure along four organizational variables: power/structure, tasks, resources, human factor • Attendance logs and observers’ ratings of occurrence and quality for expectations • 1 Core Team; 2 AAOs; 12 school teams

28. Monitoring and Mentoring Feedback Loop Chief Academic Officer Mentoring Monitoring Effective Mentoring Data from Monitoring Core Team Area Academic Officers & Lead Coaches Teacher Practices School Coaches / Principals

29. Fidelity of Whole Day First Grade Program Implementation Checklist Excerpt

30. Patterns of Quality of GBG Implementation in 12 Intervention Schools: 2004-05 Level of Implementation Week of Implementation

31. Patterns of Quality of GBG Implementation in 12 Intervention Schools: 2004-05 Level of Implementation Week of Implementation

32. Patterns of Quality of GBG Implementation in 12 Intervention Schools: 2004-05 Level of Implementation Week of Implementation

33. Patterns of Quality of GBG Implementation in 12 Intervention Schools: 2004-05 Level of Implementation Week of Implementation

34. Professional Development Model as Moving to Scaling Up • AIR (Carla Ford) transitioning from classroom/school based focus to multi-level professional development • Carla monitoring coaches; coaches monitoring teachers • 3 Area Lead Coaches being trained fall 2005; train teachers in spring 2006 • Developing monitoring system

35. Summary—Prevention and Service Researchers Researcher/Community Institution Partnerships Defining intervention and practices with precision Defining and measuring multi-level structure for sustaining and scaling-up practices Continual monitoring of practices deliberately linked to professional development Dual design requires equal emphasis on measurement of effectiveness and going to scale

36. Summary—Statisticians and Methodologists • Modeling mediation of teacher practices • Relationship of observational data on teacher practices and checklist of teacher practices: thick to thin data • Modeling multi-level structure data • Help us prepare for missing data in follow-up!