The skeletons in academic medicine’s cupboards

1. Avoiding waste in research:the role of public involvementIain ChalmersCoordinator, James Lind Initiative‘Putting people first in research’INVOLVE Conference Nottingham, 13 November 2012

2. The skeletons in academic medicine’s cupboards

3. Questions relevant to clinicians & patients? Appropriate design and methods? Accessible full publication? Unbiased and usable report? Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Over 50% of studies never published in full Biased under-reporting of studies with disappointing results Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation 50% 50% 50% 85% Research waste = over $85 Billion / year

4. Questions relevant to clinicians & patients? Appropriate design and methods? Accessible full publication? Unbiased and usable report? Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Over 50% of studies never published in full Biased under-reporting of studies with disappointing results Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation 50% 50% 50% 85% Research waste = over $85 Billion / year

5. Mismatch of patients’ and researchers’ priorities for osteoarthritis of the kneeTallon, Chard and Dieppe. Lancet, 2000.

6. Priority treatment outcome from a survey of patients with rheumatoid arthritis was not pain

7. Priority treatment outcome from a survey of patients with rheumatoid arthritis was not pain It was fatigue

10. New Healthtalkonline interview study A chance to share views/experiences of patient and public involvement in research If you’d like to contribute, contact louise.locock@phc.ox.ac.uk 01865 289303

11. How can patients and the public help to reduce waste in research?

12. Chalmers I. Lancet 2000;356:774

13. Conclusions of a 2002 study commissioned by INVOLVE “There was widespread support for the concept of a publicly available resource containing information about ongoing clinical trials designed for use by potential participants.”Campbell M, Entwistle V, Langston A, Skea Z (2002). Scoping study to explore the most appropriate way to produce and disseminate information on the quality of randomised controlled trials for potential participants. http://www.invo.org.uk/pdfs/pact_reportfinal0305.pdf

14. INVOLVE 6TH National Conference 11 November 2008The UK Clinical Trials Gateway: should provide information about ongoing clinical research in the NHSIain ChalmersJames Lind Initiative

15. For every ongoing trial being conducted within the NHS, the UK Clinical Trials Gateway should aim to provide access to:  a lay summary  the patient information sheet the WHO 20-item dataset the protocol, with links to the systematic review(s) showing why the trial is needed the trial website (if one exists)

16. Reliable, user-friendly informationabout specific ongoing clinical trialsis still NOT generally available

17. What is the James Lind Initiative and what has it done to try to address some of these problems?

19. 2003-The James Lind InitiativeFunded by the National Institute of Health Research and the Medical Research Council “to promote acknowledgement of uncertainties about the effects of treatments, and research to address them.”

21. Programme of work ofThe James Lind Initiative Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library andTesting Treatments

22. The UK Database of Uncertainties about the Effects of Treatments Established to publish uncertainties about the effects of treatments which cannot currently be answered by referring to relevant and reliable, up-to-date systematic reviews of existing research evidence

23. The UK DUETs Development Team Mark FentonEditor, DUETs Hazim Timimi Update Software

24. UK DUETs draws on three main sources • Patients', carers' and clinicians' unanswered questionsabout the effects of treatments • Research recommendationsin reports of systematic reviews and clinical guidelines • Ongoing research, both systematic reviews in preparation and new 'primary' studies

26. Programme of work ofThe James Lind Initiative Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library andTesting Treatments

27. Established in 2004 to see if it would work Iain Chalmers (James Lind Library) John Scadding (Royal Society of Medicine) Nick Partridge (INVOLVE)

28. To increase the focus of the therapeutic research agenda on questions and priorities shared by patients and clinicians. To promote Priority Setting Partnerships involving patients and clinicians to identify and promote their shared priorities for therapeutic research. To increase general awareness and understanding of the need to refocus the therapeutic research agenda.

29. Lester Firkins Chair, JLA Strategy & Development Group Sally Crowe Chair, JLA Monitoring & Implementation Group Katherine Cowan Editor, JLA Guidebook Patricia Atkinson Administrator, JLA Secretariat

30. Involving patients, carers and clinicians in research priority setting The JLA’s principles Inclusive Balance of perspectives Accessible to all Supportive Recognising a range of capacities and skills Transparent and democratic Data sharing Agreed protocol Declaration of interests Neutral facilitation Communication and feedback

31. JLA Priority Setting Partnerships

32. Identifying and prioritising uncertainties about the effects of treatment Harvest ‘raw uncertainties’ Stage 1 Stage 2 Derive ‘indicative uncertainties’ Publish ‘indicative uncertainties’ in UK DUETs Stage 3 Prioritize uncertainties, James Lind Alliance Priority Setting Partnerships Stage 4 Agree shortlist of highest priority uncertainties Stage 5 Notify highest priority uncertainties to research funders 19,551 2555 1678 264

33. Researchpriority themes[across asthma, incontinence, vitiligo, eczema, stroke, prostate cancer, schizophrenia, aspects of balance, and type 1 diabetes] • Assessment of long-term effects (wanted and unwanted) of treatments • Assessment of safety and adverse effects of treatments • Assessment of complementary and non-prescribed treatments • Assessment of strategies to improve early diagnosis and treatments, and harmonisation of practice • Assessment of the effectiveness and safety of self-care

34. For more information… www.lindalliance.org – become an Affiliate www.JLAguidebook.org – practical guidance @LindAlliance

35. From 1 April 2013, the JLA will be incorporated in the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) Tom Kenny, Pamela Young, Sarah Fryett

36. Questions relevant to clinicians & patients? Appropriate design and methods? Accessible full publication? Unbiased and usable report? Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Over 50% of studies never published in full Biased under-reporting of studies with disappointing results Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation 50% 50% 50% 85% Research waste = over $85 Billion / year

37. Publication (2007) after registration (1999) Country Size Phase Funder Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009). Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 6(9): e1000144.

38. “Studiesthat report positive or significant results are more likely to be published andoutcomes that are statistically significant have higher odds of being fully reported.” PLoS ONE, August 2008;3:e3081

40. Alessandro Liberati

41. Because research results have not been made public……patients have suffered and died unnecessarily and resources for health care and health research have been wasted.

42. TGN1412 TGN 1412

45. Why is medical academia content to acquiesce in biased under-reporting of research?