Identifying Group Characteristics Governing Outcomes in Two ...

1. Identifying Group Characteristics Governing Outcomes in Two Treatment Groups From A RCT: A Qualitative/Quantitative Study

2. Society for Adolescent Substance Abuse Treatment Effectiveness Annual Meeting Brett C. Engle, M.S.W. Doctoral Student Email: bretton.engle@fiu.edu   Mark J. Macgowan, Ph.D. Associate Professor Email: macgowan@fiu.edu   Eric F. Wagner, Ph.D. Associate Professor, Director Email: wagnere@fiu.edu Community-Based Intervention Research Group Florida International University, University Park, MARC 310 11200 SW 8th Street Miami, FL 33199

3. Acknowledgments This research was made possible through grant R01-AA10246 funded by NIAAA.

4. Outline for Presentation Introduction Study aims Process research Iatrogenic effects of group interventions with high-risk adolescents Burlingame, MacKenzie, & Strauss’s (2004) framework Methods TIP: The original RCT study Present Study: Secondary Data Analysis Measures: completed by PI and independent observer Findings formal change theory patient (or client) characteristics group structural factors small group process the group leader Summary Limitations Conclusions

5. Study Aims This study describes the need for process research on adolescent treatment groups, particularly with high-risk youth. A heuristic framework is presented to help identify and evaluate potential factors that contribute to outcomes. Qualitative and quantitative methods are employed to inductively explore and describe factors that may have been particularly important in contributing to outcomes in the two reviewed groups.

6. Process Research Process research (PR) evaluates an intervention through a systematic method of identifying and analyzing intervention components; it elucidates which intervention elements contribute to what outcomes (Bukoski, 1991). PR describes or predicts theoretical mechanisms of change operative within the intervention (Burlingame, MacKenzie, and Strauss, 2004). Kazdin and Nock (2003, p. 1117) contend that “the study of mechanisms of treatment is probably the best short-term and long-term investment for improving clinical practice and patient care.” PR is particularly warranted in studying interventions that are popular, yet may produce iatrogenic outcomes, such as group interventions with high-risk adolescents.

7. A Heuristic Framework for Examining Group Interventions Burlingame, MacKenzie, and Strauss (2004) developed this framework to identify forces influencing outcomes in group interventions: formal change theory, i.e., quality assurance or fidelity group structural factors, i.e., setting and group composition patient (or client) characteristics, i.e. diagnoses and demographics small group processes, i.e., interaction patterns, group climate, cohesion, and deviancy training the group leader, i.e., behavior and skills

8. Methods

9. The Teen Intervention Project (TIP) Original Study TIP was a randomized clinical trial (RCT) to test the efficacy of a standardized 10-session school-based substance abuse treatment group. The TIP treatment was designed to reduce substance use and related consequences among middle and high school students with substance use problems. Pretest/posttests included Tarter’s Drug Use Screening Inventory-Revised (DUSI) and Sobell’s Timeline Followback- for several substances.

10. Present Study This study is a secondary data analysis on two TIP groups using qualitative and quantitative methods. Group selection… Second author selected these two groups, because they had the greatest pretest to posttest changes on several substance use variables on the DUSI-R. A Wilcoxon nonparametric test was employed and rejected the null hypothesis that there was no significant difference in scores from pretest to posttest. The statistically significant posttest outcomes for these two groups were in the negative direction according to several variables of the DUSI-R and TLFB. e.g., Students drank significantly more drinks per drinking day after the intervention. e.g., Students were abstinent a lower proportion of days following the intervention.

11. Present Study (cont.) Data consisted of 15 out of 16 total group session audio recordings, which PI reviewed and transcribed partially or completely. Qualitative analyses involved reviewing recordings and transcripts and identifying themes, codes, and patterns, which were later organized under the Burlingame, MacKenzie, and Strauss, (2004) framework. Quantitative data included… PI ratings of group processes, using the Group Climate Questionnaire Scoring (MacKenzie, 1983). The Harvard Community Health Plan Group Cohesiveness Scale (Budman et al., 1987). The Leader Adjective Measure (MacKenzie et al., 1987), used to quantify leadership behaviors. Fidelity was rated by an independent observer and reviewed by PI.

12. FINDINGS

13. Formal Change Theory: TIP Treatment Problem behavior theory posits that multiple domains contribute to the initiation of substance use problems, and intervention efforts (Jessor and Jessor, 1977). Social learning theory targets the most potentially modifiable environmental influences and learned beliefs and behaviors associated with substance use (Wagner, Myers, and Brown, 1994)

14. Formal Change Theory TIP encourages participants to continually reevaluate expectancies about the effects of substances, believed to mediate use (Wagner, Kortlander, and Morris, 2001). TIP also utilizes motivational interviewing techniques that recognize the importance of the adolescents’ motivation for change and further encourages them to assess their own use and set meaningful goals (Wagner, Kortlander, and Morris, 2001; Miller and Rollnick, 1991).

15. Formal Change Theory: Fidelity Review Manual-based and highly structured An independent observer rated leader completion of approximately 160/175 tasks and concepts on a 5 point Likert scale and 14/18 concepts dichotomously, i.e., yes/no. The principal investigators also reviewed the group leader’s adherence to the TIP curriculum concepts as outlined in Wagner, Kortlander, and Morris, 2001 and estimated that the group leader covered 95% of curriculum content, but several concepts could have been covered in more depth. Due to less than 100% attendance and complete coverage of all curriculum concepts, not all youth received the full dosage of formal change theory.

16. Client Characteristics All participants were screened into the original study using the Composite International Diagnostic Interview and were identified as having abused alcohol or other drugs. Additional client characteristics as they relate to group composition are described under group structure, next.

17. Group Structure Attendance Both of the groups studied had a modal attendance rate of five students per session with up to three students attending two or fewer sessions. The five regular attending students missed no more than three sessions. Composition Members ranged from 13 to 15 years of age. The two groups were comprised of students from two different regular middle schools. Group A’s core members were four girls and one boy, while group B consisted of three girls and two boys. Group A included four White students and one Latina student. Group B was comprised of six White students and one African American Student, who attended fewer than half of the sessions.

18. Group A:Attendance, Demographics and Diagnoses

19. Group B:Attendance, Demographics and Diagnoses

20. Group Structure:Summary Each group contained 5 core members, the majority of whom were girls. Most students had a diagnosis of CD and several had additional diagnoses. Most students identified marijuana as their drug of choice; others students did not specify a drug of choice.

21. Group Process: Qualitative Observations The interaction patterns of the two groups were similar in that discussion was primarily content related and occurred between the group leader and individual group members. More disruptive/off task behaviors were noted in group B and could be largely attributed to two members, who exhibited very distinct and unrelated behaviors. Limited exploration of individual issues disclosed in group was noted.

22. Group Process: Quantitative Measures Group Climate Questionnaire Scoring (MacKenzie, 1983) scores. Engagement: positive working group atmosphere Conflict: anger and tension in group Avoiding: avoidance of personal responsibility of group work by the members. Scores range from 1 to 6

23. Group Process Quantitative Measures (cont.) The Harvard Community Health Plan Group Cohesiveness Scale (Budman et al., 1987). Higher scores are considered more positive. Six dimensions: fragmentation vs. cohesion; withdrawal vs. interest; distrust vs. trust; disruptiveness vs. cooperation; unfocused vs. focused; abusiveness vs. caring. Scores range from -5 to 5.

24. Group Process:Summary The two groups were overall very similar in that their interaction patterns were primarily content related and individually directed towards the group leader. Group B exhibited more disruptive behaviors, due to one distracted member and one low status norm breaking member, which likely influenced climate and cohesion rating scores. Potentially important member issues were discussed only minimally and almost not at all by group members.

25. Group Leader:Qualitative Observations Leader was a white, female, psychologist with previous group work experience with adolescents. She based group discussion predominately on curriculum content. She seemed to effectively prevent or avert much rule breaking and subsequent peer reinforcement behavior, i.e. deviancy training by keeping focused on curriculum content. She verbally redirected students’ attention at least three times more often in Group B than Group A. Most such attempts were in response to inattentive rather than overtly destructive behaviors. She expressed acceptance and caring but was directive and openly biased, expressing preference towards reduction and cessation of substance use.

26. Group Leader:Quantitative Measure The Leader Adjective Measure (MacKenzie et al., 1987). Scores may range from 20 to 210; lower scores have been associated with more positive outcomes in previous studies. Measure was completed by PI, based on observations of all 15 available group sessions. The group leader scored 69 on the Leader Adjective Measure for Group A and 66 for Group B.

27. Group Leader:Summary Leader references to the “here and now” were rare and leader questions posed to the entire group regarding individual member behaviors or issues were rare. She maintained a high level of control and structure, as evidenced by the group members’ overall compliance with leader instructions and requests and the infrequent need for leader reminders of group rules, especially in group A. She explored individual member issues or challenges only briefly.

28. Study Summary Process research is lacking and needed to understand the mechanisms of change, particularly in interventions with the potential to do harm. This line of research is very much in the exploratory stages with a few notable exceptions, i.e., Dishion, Poulin, and Burraston, (2001). Burlingame, MacKenzie, and Strauss’s (2004) model may be useful in studying such mechanisms in group interventions, as many factors likely influence outcomes. 

29. Limitations Although the data for this study comes from a RCT, no causal inferences can be made from this secondary data analysis. The negative outcomes of the groups cannot be attributed to the intervention at this point. The quantitative process measures were based on ratings by the PI only, which is not ideal.

30. Conclusions This study presents a model and combines qualitative and quantitative methods to suggest a preliminary approach to process research on adolescent group interventions. Additional studies, using much of the same methodology with larger samples of intervention groups with positive, neutral, and even negative outcomes will likely yield important findings regarding the mechanisms of change and/or characteristics of such group interventions.