Mark W. Fraser, PI, School of Social Work, UNC-Chapel Hill Steven H. Day, School of Social Work

1. Social and Character Development in Elementary School: The Effectiveness of the Making Choices Program Mark W. Fraser, PI, School of Social Work, UNC-Chapel Hill Steven H. Day, School of Social Work Shenyang Guo, School of Social Work Alan Ellis, Sheps Center and School of Social Work Roderick A. Rose, School of Social Work Maeda J. Galinsky, School of Social Work Kim Dadisman, Co-PI, Center for Developmental Science, UNC-CH Dylan Robertson, Center for Developmental Science Tom Farmer, School of Education, Pennsylvania State University Preliminary Findings This presentation was given at the School of Social Work, University of Maryland, Baltimore, MD, on April 9, 2009. Portions of this report were presented at the annual meeting of SACD Project grantees on June 13, 2008 in Washington, DC

2. Agenda Theoretical Bases and programs Design and challenges Analytic strategies Analytic methods (skim – see slides) Findings

3. Acknowledgments This project was support by a cooperative agreement (R305L030162)with the Institute of Education Sciences at the U.S. Department of Education (US DOE). Funding for the project was appropriated by the US DOE and the Centers for Disease Control and Prevention. We thank Paul Rosenbaum (U Penn), Ben Hansen (U of Michigan), and Matthias Schonlau (Rand Corp) for their consultation on methodological issues related to this presentation.

4. Teachers Talk about Making Choices • Changes in Classroom Atmosphere • Observable Differences in Student Behaviors • Measurable Academic Achievement

5. Classroom Atmosphere “I noticed that the classroom started working more as one big group instead of individuals.” Gr.5 Sandy Grove Elementary, Hoke County

6. Observable Behaviors “The students tend to be less critical of each other and more understanding of each other’s differences.” Gr. 5 Sandy Grove Elementary, Hoke County

7. Academic Achievement “ The program uses excellent books to support the goals of being a good friend and not hurting others.… I use them during Language Arts time.” Gr. 4 Tommy’s Road Elementary, Wayne County

8. “It provided a way for students to put their feelings into words.” Gr. 2, Bunn Elementary, Franklin County Observable Behaviors … I am feeling really mad!

9. Academic Achievement “My students spend more time on task. They seem less distracted by annoying behavior.” Gr. 5 Scurlock Elementary, Hoke County

10. Children are actually stopping and thinking about making the right choices, and I have heard a lot of children say to themselves, “Make the right choice.” It is great to hear. Kdg. Bunn Elementary,Franklin County Make the right choice!

11. “It made a difference with teaching children how to deal with their feelings using better methods rather than having tantrums or hitting.”Kdg. Bunn Elementary, Franklin County Oh, boy! I need that Making Choices program.

12. Classroom Atmosphere “This program provided a foundation on which we could build a classroom community.” Gr. 1 North Drive, Wayne County

13. Children in my school need… …social skills to make friends and deal with interpersonal problems. …lessons that teach children respect toward others and responsibility for their own actions. …a program designed to reduce disruptive behavior and promote academic achievement. Making Choices

14. Does social and character education work? (i.e., is Making Choices effective?) Research Question:

15. Intervention Research Perspective: The Design and Development Approach Specify the problem and develop a program theory Create and revise program materials Refine and confirm program components (sequential experimentation perspective) Assess effectiveness in a variety of practice circumstances and settings Disseminate findings and program materials Source: Fraser, M. W., Richman, J. M., Galinsky, M. J., & Day, S. H. (2009). Intervention research: Developing social programs. New York, NY: Oxford University Press.

16. Social and Character Development in Childhood: A Risk and Resilience Orientation PROBLEM THEORY:Perspectives on Conduct Problems and Academic Achievement in Elementary School Developmental risk perspective Ecological theory Social information processing theory

17. Eco-Developmental Risk Cascade Elementary School School Entry Pre-School Peer Rejection Academic Failure Parenting Family-School School Climate Neighborhood Biological Risks Parenting Family-School Pre-School Climate Neighborhood School Readiness Processing Skills Parenting Family-School School Climate Neighborhood Increasingly Broad Repertoire of Potentially Damaging and Aggressive Behaviors POINT: Risk factors for poor developmental outcomes vary over time. Lacking effective intervention, the potential for poor outcomes increases – and cascades – as function of complex bio-social processes. To promote positive outcomes, we must disrupt malleable risk mechanisms.

18. Cognitive Mediation Model(in Developmental Sciences) Biological Predisposition Biological Predisposition • Parenting • Monitoring • Bonding • Mental Processes • Social knowledge • Scripts • Schema/skills • Conduct Problems • Conduct disorder • Fighting • Drug use • Peers • Deviancy training • Contagion effect • False consensus • effect • Sociocultural Context • Stress/poverty • Racism • Street codes • Acute/chronic stress • Sociocultural Context • Stress/poverty • Racism • Street codes • Acute/chronic stress Adapted from: Dodge, K. A., & Pettit, G. S. (2003, p. 351). A biopsychosocial model of the development of chronic conduct problems in adolescence. Developmental Psychology, 39(2), 349-371.

19. Arousal, Emotions, Social Knowledge Social Information Processing Theory: SIP Skills and Emotional Regulation as Malleable Mediators? State the problem Interpret social cues Set goal(s) Generate potential solutions Encode social cues Assess outcomes Evaluate potential solutions Select & enact the best solution(s) Social Knowledge: Life experiences producing scripts, schemata, skills, and beliefs

20. PROGRAM THEORY (specifies how a program is to work)

21. Intervention Program Structure Such as: Targeting Unit: Classroom Entire school Other (after school, family) Curriculum Structure: Distinct activities Embedded in curriculum Activities to address SACD Goals Such as: Character education Violence prevention/peace promotion Social and emotional development Tolerance and diversity Risk prevention and health promotion Behavior management Social and Character Development: Prevention Model Behavior Positive Behavior Responsible behavior Prosocial behavior Self-regulation Cooperation Negative behavior Aggression Minor delinquency Disruptive classroom behavior Social - Emotional Competence (mediator) Attitudes about aggression Self-efficacy Empathy School Climate (mediator) School connectedness Victimization Feelings of safety at school Parent involvement Academics Academic competence School engagement Grades Standardized test scores Moderating Factors ChildFamilyCommunity Gender Parenting practices Community risk factors Socioeconomic status Home atmosphere Social capital Race/ethnicity Risk status ProgramSchool Prior test scores/grades Fidelity Activities to promote social and character development Intensity and dosage Organizational structure

22. Social Development Model Perspective

23. PROGRAMS

24. The Competence Support Program* Social DynamicsTraining for teachers Social Skills Training for students Group randomization: Cohort 1: Hoke and Wayne Counties (10 schools randomized to 5 intervention; 5 control) Cohort 2: Franklin County (4 schools randomized to 2 intervention; 2 control) Classroom Behavior Management Trainingand Consultationfor teachers *Developed by the program investigators, the intervention simultaneously focuses on the characteristics of children and on the classrooms in which they learn. The intervention combines three components.

25. Program Elements Making Choices: Skills Training curriculum for students in elementary school. In-service training introduced teachers to the risks of peer rejection and social isolation, including poor academic outcomes and conduct problems. Throughout the school year, teachers received consultation and support (2 times per month) in providing lessons designed to enhance children’s social information processing and other skills. As a part of the Standard Course of Study, the program was integrated into routine class instruction. Classroom Behavior Management provided teacher consultation on classroom management strategies designed to strengthen engagement in instructional activities. Social Dynamics Training provided teacher consultation on classroom contexts, social groupings, and interactional patterns that can be used to reinforce academic achievement and prosocial behavior.

26. Theory of Change: Making Choices Core #1 Core #2 Core #3 Core #4 Core #5 Random Assignment Application of Making Choices by Teacher or Counselor SIP skills of the Children in the School Impact on Social Engage-ment and Peer Rejection Training the Teacher or Counselor Impact on Disruptive Behavior and Academics • Test the degree to which the intervention is delivered as intended, e.g., specific activities • Assess implementation of training • Assess if teacher acquires skills from training/supervision Characteristics of the Teacher or Counselor Characteristics of the Children and the Classroom Note. In a randomized trial, you must figure out a way to measure each of the core elements. Treatment as Usual Control Condition

27. Make Program Manuals From risk mechanisms, mediators, and logic models to the design of a program Specifying program activities that target the malleable mediators and have cultural congruence Example: Making Choices For a discussion of issues in the development and use of treatment manuals, see: Galinsky, M. J., Terzian, M. A., & Fraser, M. W. (2006). The art of group work practice with manualized curricula. Social Work with Groups, 29(1), 11-26.

28. Warning: It is easy to under estimate the difficulty of developing a program manual.

29. “That Sunk Feeling” If you start in the wrong place, it usually does not help to dig deeper! Source: Don Moyer, Harvard Business Review (October, 2004, p. 160)

30. Start with theory and research, plus practice experience… Develop a template for each lesson or session How to begin in the right place…

31. Grade 2 Lesson 2 Overview Standard Course of Study Prep Materials Activity 1 Review Answers Prop Process Tip

32. Avoid labeling Scenarios Activity 2: Write About It!

33. Develop all worksheets and artwork

36. “Pete the Penguin” Poster for Grade 2

37. Sample Lesson Activities from Making Choices

38. Grrrrr! MEAN FRIENDLY CAN’T TELL Gr. 3 Lesson - Intentions SYMBOLS

39. LAUGHS HITS SHARES WITH YOU WITH YOU YOU MAKES HUGS IGNORES Grrrrr! CAN’T TELL A FACE YOU YOU AT YOU MEAN FRIENDLY FRIENDLY FRIENDLY TALKS BITES HELPS ABOUT YOU YOU YOU Intentions: Mean or Friendly? Grade 3 Lesson 10

40. GOAL SETTING GOAL: Something a person wants or something a person wants to see happen. RELATIONSHIP GOAL: Goals that involve wanting to get along with another person. Grade 4 Lesson 6

41. Are these Relationship Goals? (thumbs up or thumbs down) • I want to make an “A” on my math test. • I want to play more often with my friend. • I want a new video game for my birthday. • I want to eat out at a restaurant for dinner. • I want to become friends with the new student. • I want to join in the basketball game at recess. • I want to sit with Jose on the bus. • I want to be in the class play this fall. • I want to stop getting upset when friends ignore me.

42. GOAL SETTING • Set a relationship goal for these situations: I was playing basketball at recess with some friends. Terrell, who is not very good at basketball, asked if he could play with us.

43. Set a Relationship Goal Denise just made me really upset. She tried to pick a fight with me by saying things that are not true. I am feeling angry with her right now.

44. Set a Relationship Goal Yesterday, my mom gave me a really cool pen that writes in all different colors. When I brought it to school this morning, Stacey asked me if she could borrow it. Last time I let Stacey borrow something she lost it, but if I say no she might get angry with me.

45. EVALUATION DESIGN: Cluster Randomized Trial with Ten Schools Randomly Assigned to Treatment (j=5) and Control (j=5) Conditions Cohort Design: Intervention provided in grades 3, 4, and 5 Prior Studies Single-group qualitative trial of MC intervention (8th grade girls) Two-group cluster randomized trial at classroom level in one middle school (6th grade only) Two-group cluster randomized trial at classroom level in one school (3rd grade) Two-group, MC+SF intervention randomized trial (11 sites, 3rd – 4th grade) Cohort sequential study by classroom in two schools (3rd grade) (Current) Two-group cluster randomized trail at 14 elementary schools

46. SAMPLE

47. Two Overlapping Samples Grade 3 n=571 Grade 4 n=557 • Grade 3-4 Sample • 3rd and 4th graders • 10 schools • Any consented students on a 3rd or 4th grade roster • Change=entrants-leavers Grade 5 n=433 n=414 n=370 • Grade 3-4-5 Sample • 3rd, 4th, and 5th graders • 9 schools* • Only consented students on a 5th grade roster • Change=addition of entrants * One treatment school was reorganized into a different building and dropped the program between 4th and 5th grade; students from that school were excluded from the 3-4-5 sample.

48. Equivalence of Intervention and Control Groups on Selected Child, Family, and School Attributes: Grade 3 Cohort 1

49. Difference in School-Level Academic Performance: Percentage at Grade Level Test results for 2005-06 and 2006-07 are based on a revised accountability model and are not comparable to those from previous years.

50. Sample sizes vary because pretest measures were collected from different respondents (teachers, students) at different times. SLA and Peer assessment pretest were collected from students at the end of 2nd grade. CCC and ICST were collected from teachers at the beginning of 3rd grade. SLA=Skill Level Assessment (SIP skill – HOME Scale adaptation by Dodge, 1980). CCC=Carolina Child Checklist (Macgowan et al. 2002 – Research on Social Work Practice). ICST-Interpersonal Competency Scale – Teacher (Xie et al., 2002, Social Development)