1 / 43

Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood

Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood Health Care of Older People Nottingham University Hospitals. What we may be witnessing is not just ... the passing of a particular period of post-war history, but the end of history as such...

edison
Download Presentation

Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood Health Care of Older People Nottingham University Hospitals

  2. What we may be witnessing is not just ... the passing of a particular period of post-war history, but the end of history as such... That is, the end point of mankind's ideological evolution and the universalization of Western liberal democracy as the final form of human government.

  3. The end of Medical History John R Hampton

  4. Examples of complex services for older people • Rehabilitation • (stroke, community hospitals, intermediate care) • Most psychiatry • Falls services • Continence services • Care home and long term care interventions • Education, training, guidelines, care pathways, service improvement

  5. Complex interventions • intervention … comprising a number of separate elements which seem essential to its proper functioning; • components … may act independently and inter-dependently • usually include: • behaviours • parameters of behaviours (frequency, timing) • methods of organising and delivering those behaviours (types of practitioner, setting and location). MRC 2000

  6. Plan • Briefly describe 4 trials • Highlight design, implementation, analysis and interpretation issues • Consider design compromises in a new trial of a complex intervention constrained by service demands

  7. Stroke Unit Evaluation 1991-3 • Operational since 1982, 15 beds • Restrictive inclusion criteria • 315/1760 (18%) stroke patients included • SU median 76 days, standard care 54 days • Follow up: 275(87%) at 3 mo; 241 (77%) at 12 mo • At 12mo: • Died 14% vs 19% • Home 65% vs 62% • Barthel 17 vs 16 • Rivermead ADL 12 vs 9 • NEADL 7 vs 4 • GHQ 17 vs 20 • psychological adjustment better at 3 & 6 months

  8. Stroke Unit Evaluation issues • Keep unit full (changed to 60:40 randomisation) • Blinded assessment • Loss to follow up (23% at 12 months) • Contamination • Power • Multiple outcome measures • … but what is a stroke unit? • … why the difference in outcome?

  9. SUE observation of process • Non participant observations of 76 patients • 3x8h shifts over 3 days, 10min intervals • location • rehab 6.4% vs 3%; bedside 61% vs 65%; social 5% vs 2% • activity • isolated disengagement 15% vs 24% • positioning - poor 13% vs 30% • contacts • more time with nurses, therapists; less time with doctors, visitors • rehab time (on ward) 46 min/d vs 23min/d

  10. Community Stroke team • Established multidisciplinary service • 428 referrals randomised (50:50 then 40:60) over 2y • Group A: routine care including day hospital or OP (n=189) • Group B: CST (n= 232). Median 18 sessions (range 0-115, IQR 10-42). Mean 4.8 h PT; 3.8h OT; 2h SLT, 1.9h MH nurse; 0.5h Rehab Support • Six months later, contacted by letter, requested consent to follow up, postal outcome questionnaire • Consent procedure: • avoid perception of ‘missing out’ by control group • reduce team loyalty bias • logistically easier • Offer of assistance or visit to complete questionnaire

  11. Community Stroke Service trial: issues

  12. Outcomes at 6 months Patient CST vs standard Barthel/20 16 vs 16 NEADL/44 24 vs 26 GHQ/24 13 vs 15 EQ5D/100 52 vs 55 Satisfaction emotional support/4 3 vs 2 (CST better) overall/4 3 vs 2 (CST better) Carer GHQ/24 13 vs 15 Carer strain/12 8 vs 10 (CST better) EQ5D 73 vs 75 satisfaction knowledge/4 3 vs 2 (CST better) overall/4 3 vs 2 (CST better)

  13. Community Stroke Team: problems • Low and differential follow up(47% vs 43%) • No baseline data • No record of routine services • Time but not content of CST intervention recorded • Was the service successful or not? • If this didn’t work what could …?

  14. EDRS: service structure A new EDRS was established in 1998. 9-month development phase. Staffing: 2 OTs, 2 PTs, 3 nurses, 1 SW, 7 rehab assistants, secretarial support. No doctors. Visits for assessment, monitoring, rehabilitation, or assistance and care. Up to four visits per day, up to 7 days per week, 8am to 10pm. Duration up to 4 weeks.

  15. EDRS: randomised trial • 370 elderly medical, stroke and surgical patients • Randomised between EDRS and conventional care • Mean 22 visits per patient over 4 week • Mean 7 days shorter hospital length of stay

  16. EDRS: outcomes

  17. Qualitative evaluations • Exploratory, in depth • Subject to ‘rules’ • Identify: • what an intervention comprises in practice • how an intervention is useful • how it achieves its effect • the conditions necessary for its effect • Introduce rigour into otherwise potentially anecdotal evidence

  18. EDRS: Qualitative study • Patients interviewed: • before hospital discharge • after 4 weeks • after 3 months. • Staff responsible interviewed 6–8 weeks later • 64 interviews of 20 patients • 15 interviews of 11 staff

  19. Qualitative study • Before discharge • Participants anxious about return home, but keen to do so • Felt vulnerable, feared not being able to cope, or being a burden on families. • After discharge • Patients felt they had improved • … but described pain, loneliness, fatigue, reduced mobility, problems climbing stairs or getting outdoors. • Lack of confidence and anxiety about activities

  20. Qualitative study • EDRS delivered: skilled assessment, negotiated treatment goals meaningful to the patient, and demonstrated team working. • Interventions: functional rehabilitation, teaching skills, information giving, advice, overcoming emotional barriers, provision of aids and appliances and provision of personal and domestic care. • Emotional support: improved confidence and morale, and led to greater task performance and psychological well-being.

  21. Delirium prevention trial • 1 intervention ward, 2 standard care wards • Multi-component delirium prevention regimen • hydration • vision and hearing • orientation • mobilisation • non-pharmacological sleep promotion. • RCT pilot failed - ‘120% bed occupancy’ • Prospective individual matching design

  22. Prospective individual matching design • Prospectively assessed: 2424 patients admitted to 3 wards • Research staff ‘not involved in care’, ‘unaware of nature of study, or patient group assignment’. • Over 70, at intermediate or high risk of delirium • Recruited all patients on intervention ward • Matched these with standard care patients •  5 years • sex • baseline risk of delirium • U&E, visual and cognitive impairment, APACHE-II.

  23. Exclusions • Assessed 2424 patients • Excluded 1265 • severe dementia 154 • aphasia 38 • no English 92 • coma or terminal illness 69 • LOS<48h 219 • other incl. no interviewer or patient unavailable 355 • Declined 250 • No match 67 (24 intervention, 43 standard) • Recruited 852 (426 intervention, 426 standard)

  24. Results

  25. Issues • Able to do research interviews on all at admission • Demonstrated power equivalence of RCT • Recruitment • Demonstrated good matching at baseline on 32 variables • Contamination • Bias and blinding

  26. MRC 2000 framework

  27. MRC 2008

  28. NIHR Medical and mental health unit controlled clinical trial

  29. Medical and mental health unit: development • Initial 250 patient register/cohort study. • Multi-professional operational group • Additional staff: RMNs and MH specialist OT/PT • Programme of staff training • Advice from MH service • Environmental changes • New approach to families • Develop policies and new working practices

  30. Medical and mental health unit ‘sufficiently different from standard care to have a reasonable chanceof demonstrating different outcomes in a trial’

  31. Medical and mental health unit: controlled clinical trial • Modelled on stroke unit trials • Compared with ‘standard care’ • Starting May 2010 (pilot); July 2010 (main trial) • up to 2 year recruiting, 3 months follow up • Target n=240 per group • Effectiveness, economic and qualitative studies: • not just does it work, but how does it work? • is it value for money? • Service funding ceases April 2013

  32. Medical and mental health unit: controlled trial • Outcomes at 3 months: • number of days spent at home: • length of stay, readmissions, deaths, care home placements • number of days in care home for those starting there • scales: • DemQoL, NPI (behaviour), Barthel, EQ5D, London Handicap • Carer strain and psychological wellbeing • resource use and costs • non-participant observer study

  33. Medical and mental health unit: recruitment and randomisation • Constraints • No waiting on AMU (for assessment/recruitment/consent/MMHU bed) • No empty beds on MMHU • Keep MMHU full with suitable patients • Minimise ward moves for confused patients • Likely to lose trial beds to non-randomised patients

  34. Medical and mental health unit: Recruitment and randomisation • Randomisation prior to consent, recruit to follow up later. • ‘Confused, over 65’ • Web based stratification (care home or not) • Transferred to MMHU or standard ward • Once on ward recruited to trial (consecutively or randomly) • Other issues • Need to persuade AMU to do something of no benefit to them • Majority lack capacity to consent: effort and delay • 10% lack a consultee • 40% of cohort declined

  35. CLRN/MHRN to the rescue • Initial plan was limited by availability of researchers to recruit and follow up • Current plan to recruit all randomised patients who agree to take part • Option to collect primary outcome on those declining full study

  36. Summary and lessons for the triallist • The RCT can be adapted to evaluate complex interventions • Logistics can be tricky and compromises needed • Interpretation is more difficult • multiple outcome measures • what is the outcome? • does the outcome match the intervention? • condition specific vs generic outcomes • qualitative evaluation • Look at the content of the intervention • what is the intervention? • how different it is from the comparator

  37. Complex interventions Watch this space! Thank you.

More Related