DEREK MILNE, ALIA SHEIKH; SUE PATTISON; ANDREW WILKINSON

1. BABCP Annual Conference, Exeter, July 17th, 2009THE EVIDENCE-BASE FOR CBT SUPERVISION:SYSTEMATIC REVIEWS OF THE EMPIRICAL LITERATURE DEREK MILNE,ALIA SHEIKH; SUE PATTISON; ANDREW WILKINSON

2. Background Clinical supervision training regarded as essential. Yet training not been given systematic attention (“Something does not compute”: Watkins, 1997) : concerns that evidence-base generally weak, etc. One way forward: selective (BES) systematic review of controlled studies.

3. Best evidence synthesis (BES): a novel, affirming approach to the systematic literature review: • BES “combines the meta-analytic approach of extracting quantitative information in a common standard format…with a systematic approach to the assessment of study quality and study relevance” (Petticrew &Roberts, 2006, p.181); • Often used in education, to answer questions about what works (pragmatic focus); • Careful selection of studies - Milne et. al. 2006: 9 inclusion criteria, inc: ‘CS manipulated; demonstrated effectiveness; in scientific journal’.

4. Objectives • To assess the degree of empirical support for supervisor training • To define training methods that are supported in the selected literature • To specify recommendations for the training of supervisors

5. Methodology • Conducted systematic literature search of three databases (MedLine, PsycInfo, ASSIA) • Key words: “clinical supervision,” and “clinical supervisor” • Narrowed search using term “empirical” • Studies selected according to specific inclusion criteria, based on NICE(R) Manual for Systematic Reviews of the Clinical Supervision Literature (Version 11; Milne & James, 2000) • Final sample: 11 studies (N= 145 supervisors; 251 supervisees; 147 clients)

6. Methodology • Studies also evaluated according to NICE(R) (2001) coding manual • Coding system allows analysis of the methodological quality of studies (e.g., use of measures, threats to validity, etc) & of the methods (Independent variables) • 2 studies independently coded by researchers, 1 at start (inter-rater agreement=81%) & 1 at end (82%)

7. Results: 2 Questions Addressing the two questions posed by Whitman, Ryan & Rubinstein (2001): What empirical support is there for supervisor training? Which elements of such training are supported by evidence?

8. Q1: What empirical support is there for supervisor training? • 11 studies provided empirical support for supervisor training (e.g. they had sound methodological validity and an average 67% impact on the 145 supervisors’ learning) • 15 training methods- primarily: corrective feedback (all 11 studies); educational role-play & observational learning (both in 8); teaching (in 5 studies: NB: next slide arranged by frequency of methods use in training)

9. Feedback Educational role-play Modeling (live/video demonstration) Teaching (verbal instruction) Written assignments Behavioural rehearsal Providing a rationale Guided reading Discussion Educational needs assessment Direct observation Questions and answers Agenda-setting Homework assignments Quiz Q2: Which elements of such training are supported by evidence?

10. These methods were blended, to be: Multiple (mean 5.1 per study); Structured (address training cycle); Complementary (experiential learning cycle)

11. Structured approach recommended Loganbill and Hardy (1983): 3 necessary components of training: theoretical content, simulated experience, and in vivo practise with supervisees; Russell and Petrie (1994): combine didactic and experiential (“critical element in an effective supervisory training programme” ; p. 39). More recent reviews agree: Steinert et al.,2006 (systematic review; faculty); Falender et al., 2004; Kaslow et al., 2004 (consensus).

12. Example: Bambling, et.al. (2006), Psychotherapy Research, 16, pp. 317-331. N=40 supervisors (RCT: CBT or Dynamic; N= 127 therapists & N= 127 patients) Trained separately (1 day workshops: teaching about instruments & role-plays) Evaluated similarly: learning (self-rated adherence) & impact (BDI; Working Alliance Inventory; client satisfaction)

13. Discussion:Some problems with BES: Ignores negative findings? True, in order to address pragmatic questions; but mixed results, yet similar conclusions Problem of induction follows? True, but accidently coded weak/negative papers were also similar (failed to falsify); Also, BES review = falsifiable statement Generalize to non-LD/brief supervision? True, there very likely are significant differences: awaits more studies of ‘complex’ supervision

14. Conclusions & recommendations • BES systematic review = pragmatic method for addressing inconsistency between policy and practice • Results offer a ‘foundational’ evidence-base for CBT supervisor training, consistent with educational theory, best practise consensus and policies (add ‘complex’ supervision evidence as it emerges) • Mental health professionals should be prepared for the recognized specialization of supervision following the structured training format

15. Acknowledgements • This study was funded by the Higher Education Academy, Psychology Network • We are grateful to Helen Aylott, Nasim Choudhri, Christopher Dunkerley, Helen Fitzpatrick, and Sarah Wharton for their initial work on this review • Thanks also to Kathryn Mark, Barbara Mellors, and Thomas Cliffe for help in preparing the ppt

16. Authors DEREK MILNE, School of Psychology, Newcastle University, Newcastle upon Tyne, NE1 7RU d.l.milne@ncl.ac.uk ALIA SHEIKH, School of Psychology, Newcastle University, Newcastle upon Tyne, NE1 7RU a.i.sheikh@ncl.ac.uk SUE PATTISON, School of Education, Communication and Language Sciences, Newcastle University, Newcastle upon Tyne, NE1 7RU susan.pattison@ncl.ac.uk ANDREW WILKINSON, North Tyneside Primary Care Trust. Andrew.Wilkinson@northtyneside-pct-nhs.uk