Knowing what or understanding how: The role of RCTs in changing clinical practice

1. Knowing what or understanding how:The role of RCTs in changing clinical practice EFTA/AFT Congress 4th - 6th October, 2007. Glasgow Ivan Eisler Reader in Family Therapy Institute of Psychiatry, Kings College London

2. All who drink this remedy recover in a short time except those whom it does not help who all die. Therefore it is obvious that it only fails in incurable cases. Attributed to Galen 2nd century A.D.

3. Our 2 psychology colleagues maintain that the RCTs all point to CBT and there is little or no RCT research evidence for the value of systems or psychoanalytical family work.  Any advice or pointers that you could offer would be appreciated, as my colleagues here and I use psychodynamic and systems approaches because we are committed to them and believe that they work

4. Smith & Pell 2003 BMJ;327;1459-1461 One of the major weaknesses of observational data is the possibility of bias, including selection bias and reporting bias, which can be obviated largely by using randomised controlled trials. The relevance to the procedure under discussion is that individuals using it are likely to have a high prevalence of pre-existing psychiatric morbidity. Individuals who do not use it are likely to have less psychiatric morbidity and may also differ in key demographic factors, such as income and cigarette use. It follows, therefore, that the apparent protective effect may be merely an example of the “healthy cohort” effect.

5. Smith & Pell 2003 Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials BMJ;327;1459-1461 One of the major weaknesses of observational data is the possibility of bias, including selection bias and reporting bias, which can be obviated largely by using randomised controlled trials. The relevance to parachute use is that individuals jumping from aircraft without the help of a parachute are likely to have a high prevalence of pre-existing psychiatric morbidity. Individuals who use parachutes are likely to have less psychiatric morbidity and may also differ in key demographic factors, such as income and cigarette use. It follows, therefore, that the apparent protective effect of parachutes may be merely an example of the “healthy cohort” effect.

6. Debriefing “the experiences of 700 CISM teams in more than 40,000 debriefings cannot be ignored, especially so when the overwhelming majority of reports are extremely positive” Mitchell & Everly, 2003

7. Debriefing • Review of 15 RCTs of single session debriefing showed no short term difference between debriefing and control • 2 RCTs with long term follow-up showed worse outcome following debriefing (particularly in those with worse initial trauma)

8. +ve result Adopt treatment as standard practice Theoretical model Development of new treatment Open testing of treatment Randomised clinical trial Clinical observations Abandon treatment -ve result The development of 'empirically validated treatments'

9. Limitations of randomized treatment trials • Subjects • selectivity • classification • dropouts • Nature of treatments • restricted nature treatments • time limitations • common factors • Evaluation of outcome • efficacy v effectiveness • short term v long term outcome • clinical v measurable outcome • outcome from whose perspective

10. Conceptualization of therapy and model of change Evolution of clinical practice Randomised clinical trial Explanatory models of disorders Clinical practice Professional consensus views The evolution of clinical practice

11. Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. Sackett et al (1996) Evidence based medicine: what it is and what it isn't. BMJ. 312, 71-2, David Sackett

12. The practice of EBM means integrating individual clinical expertise with the best available external clinical evidence from systematic research Sackett et al (1996) Evidence based medicine: what it is and what it isn't. BMJ. 312, 71-2, David Sackett

13. Conclusions • It is important to distinguish the role of evidence in making “categorical” decision and ongoing “process” decisions • The RCT paradigm provides a useful but limited test of the efficacy of treatments • RCTs (cumulatively) can have an important effect on professional consensus which influences the allocation of resources, training as well as clinical practice • RCTs, by limiting clinical autonomy, will often lead to unexpected results which challenge beliefs about mechanisms of change and conceptualization of treatment • Changes in clinical practice require a change in the understanding of how treatments work which requires a mixture of inductive and deductive reasoning