Mark Chaffin, Ph.D. University of Oklahoma Health Sciences Center

1. Out of the Laboratories and Into the Streets—Future Directions for PCIT in a Broad Range of Field Settings Mark Chaffin, Ph.D. University of Oklahoma Health Sciences Center

2. Overview • Why pick PCIT as a model for widespread implementation? • What makes field settings different from research settings? • Using technology to make it work • Adaptation • Thinking about a services research agenda

3. Why Pick PCIT as a Model For Widespread Implementation? • Effective and Robust • Across a range of children • Across a range of problems • Disruptive behavior disorders among young children • Parent-to-child violence (physical child abuse) • Parent-child relationship problems • Adaptable across cultures • Native-American adaptations • Hispanic adaptations • Effects durable over time, generalize across settings and across children in the family • Ancillary benefits (e.g., parental depression) • Teachable-learnable

4. Kauffman Best Practices Project for Child Abuse and Neglect

5. National Child Traumatic Stress Network • 50+ centers nationally, funded by SAMHSA • The mission of the NCTSN is to raise the standard of care and improve access to services for traumatized children, their families and communities throughout the United States • PCIT is one of two major models selected for implementation

6. Other Endorsements Moving PCIT Into Widespread Implementation • Office for Victims of Crime Guidelines • APA Society of Clinical Child and Adolescent EBT Designation • SAMHSA Model Programs List (in process) • Washington State Institute for Public Policy List of Cost-Effective Models • Federal Agencies Supporting Work • NIH, CDC, SAMHSA

7. Why Pick PCIT for Widespread Implementation? • The questions is actually moot. Its been picked already • The question now is HOW to implement PCIT in the growing number of field sites wanting to pick up the model

8. Academic Settings Doctoral and post-doctoral training programs lasting a year or more Graduate students and fellows Controlled and structured Long-term training and supervision by on-site faculty, usually with extensive PCIT experience Strong evidence-based culture Stable group of PCIT experts Relatively independent of system externalities Resource rich Climate promoting professional growth Field Settings Diverse field settings, often small community-based agencies Variety of staff backgrounds Less structured and controlled Usually no access to on-site expertise Evidence-based culture? Staff turnover? Very dependent on system externalities Resources? Climate? Burn-out? What makes field settings different from research settings?

9. Reputation of Evidence-Based Treatments In Field Settings? • The “Big Lies” about evidence-based treatments • “Evidence-based practices are only developed with patients that one kind of problem—our real-world clients have multiple problems” • Fact—PCIT (and most other EBT’s) have been developed and tested with highly co-morbid and multi-problem populations • “Its therapist characteristics, not models that make treatment work” • Fact—Some PCIT therapists get better results than others, but so what? Both models and therapist factors matter—who ever claimed otherwise? • “Manuals are rigid and lock-step and don’t anticipate inevitable case idiosyncrasies” • This is why you need to learn the theory and not just the manual. Getting competent with the model is the point, not being lock-step

10. Reputation of Evidence-Based Treatments In Field Settings? • The “Big Lies” about evidence-based treatments • “Evidence-based practices are only developed for white middle-class populations and aren’t relevant for other groups” • Where have you been? Certainly not reading the PCIT research, that’s for sure • A common finding is that cultural adaptations to a treatment improve retention and engagement (which is good) more than they change outcomes—most EBT yield equivalent benefits across subgroups • Culturally competent adaptations of PCIT are found to work well—and retain the core elements of PCIT • Besides, what’s the alternative—giving minority populations untested and unsupported interventions? Isn’t this just creating the care disparities we need to be working to eliminate?

11. Making Implementation Work • Consider organizational context • Support for a sustainable PCIT program within an agency or practice • You are setting up a program not training individual therapist(s) • Minimum number of therapists • Leadership commitment • Funding base and reimbursement • Organizational climate • Is this a healthy organization or a burned out organization? • Value placed on innovation and risk taking, versus value placed on conformity and CYA

12. Making Implementation Work • Consider organizational culture • “How we do things around here” • Behavioral skill focus • Hard to get PCIT buy-in among staff committed to “resolving underlying issues” • Level of therapist activity • PCIT, like most EBT, require much higher level of therapist preparation and activity—may be very unfamiliar for go-with-the-flow or free-styling therapist cultures • The “poly-therapy” culture • A therapy for every problem—emerging evidence suggests that this reduces the effectiveness of parenting programs

13. Making Implementation Work • Consider externalities and practicalities • Referral • Referral flow established • Practices/attitudes of child welfare system or court systems • Acceptability of PCIT for referral systems • Will it fit with the way service plans are drafted by CW and court? • Transportation, especially where children are in foster care • Funding (sufficient to support failed appointment costs) • Setting up a structure for make-up sessions

14. Making Implementation Work—Beyond the “Train and Hope” Approach From Joyce and Showers meta-analysis, 2002

15. What Happens After Training? • Hurlburt follow-up of TIY trainees • Phone interview of trainees who, after training to criteria, returned to field practice settings • Very few were implementing the model as designed • A high percentage had dropped key elements of the model—such as behavioral rehearsal of parenting skills • Our anecdotal experience at OUHSC is that significant drift from the PCIT model occurs despite strong commitment from field practitioners to do it right, and that this drift is not apparent in verbal consultation (the practitioner believes they have it right, and it sounds right in consultation)

16. How Can Technology Help Us With Ongoing Coaching? • Remote real-time (RRT) consultation • Designed to approximate the usual PCIT co-therapy supervision approach • Internet-based teleconferencing technology • Allows consultant to: • Work directly with a PCIT trainee from anywhere • View PCIT session in real-time • Directly coach the PCIT therapist as the session is in process • Take over coaching the parent to demonstrate • Allows the PCIT therapist to: • Talk privately with consultant about session as it is in process • Compare DPICS codings • Receive feedback on coaching as it occurs

17. What About“Drift” / “Adaptation” • What is the difference between drift and adaptation? • “Drift is what you do, adaptation is what I do” • The dilemma • Drift, where evaluated, usually results in poorer outcomes • Adaptation, to some extent, often is necessary to make things work in some settings • Adaptation, if done thoughtfully and well evaluated, can lead to growth, enrichment and development of a model • We shouldn’t tell field practitioners— “value innovation, value continuous quality improvement, but don’t try it yourself—leave it to us academics”

18. The “Drift” / “Adaptation” Dilemma • A proposed distinction • Adaptation involves field practitioners in collaboration with model experts • Perhaps additionally in collaboration with input from consumers • Adaptation stays within the overall theory model—it doesn’t mix-and-match ad hoc • Adaptation is evaluated to see if it is achieving its intended goals

19. Thinking About a PCIT Services Research Agenda • Laboratory work will continue to be important • But, it is time to begin moving much of our research focus into the services area—field based • Questions not so much about “can it work,” but “how do you make it work in diverse settings” • Questions about how PCIT will continue to evolve to suit field settings and improve field outcomes

20. Evidence-Based Practice in the Field • Broadly, Evidence-Based Practice (EBP) is not simply a group of Evidence-Based Treatments (EBT). • It is a process for how treatments are developed, tested, adapted and how they evolve • All treatments evolve—the question is about the form of natural selection that guides this evolution • EBP is a form of natural selection based on rigorous outcome testing, rather than the usual ideology-based processes guiding treatment development and evolution

21. A PCIT Evolution Agenda • Partnerships between • Those who develop the service model • Those who evaluate outcomes • Those who deliver PCIT in the field Developers Front-Line Practitioners Outcome Researchers

22. What Questions Can Partnerships Answer? • How Do We Optimally and Efficiently Train and Implement PCIT? • What Modifications Work and With Whom? • Group models? • Home-based? • Cultural Adaptations? • What Practices Can Improve Retention, Adherence to Homework, etc. in Field Settings? • What Training Backgrounds Are Necessary to Do PCIT Well? • What Happens When PCIT is Integrated Into Overall Service Packages (e.g., Family Preservation)?

23. What Do We Need to Realize This? • Consistent data, across sites • Outcomes • Fidelity and treatment content • Data at multiple levels • Organizational • Individual Therapist • Client • Willingness of • Researchers to accommodate less than laboratory levels of internal controls • Practitioners to accommodate needs for consistency in data and practices (no “free-styling” practice) • Willingness of policy makers and service delivery funders to support rigorous outcome evaluation and quality improvement efforts—not just funding “units of service”

24. Implementing EBT—THE DREAM IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY WILL COME

25. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY WON’T KNOW

26. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY MAY LIKE THE ONE THEY BUILT BETTER (even if it doesn’t work….but, they may not know this)

27. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY WILL TRY TO COME, BUT THE MAP YOU SENT WASN’T WRITTEN IN THEIR LANGUAGE

28. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY MAY WANT TO COME, BUT DON’T HAVE THE TIME

29. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY MAY WANT TO COME, BUT CAN’T AFFORD ADMISSION

30. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY MAY WANT TO COME, BUT YOU DON’T HAVE ROOM

31. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY WILL TRY TO COME, BUT END UP NEXT DOOR

32. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY WILL TRY TO COME, BUT YOU FORGOT TO BUILD A GATE TO LET THEM IN

33. Implementing EBT—THE REALITY IF YOU BUILD IT………. Thanks to Jim Emslie and Linda Anne Valle THEY WILL COME…… THEN DECIDE TO REMODEL IT UNTIL IT NO LONGER RESEMBLES WHAT YOU BUILT

34. Implementing EBT IF WE BUILD IT JOINTLY……. THEN WE’RE BOTH ALREADY HERE