ILCOR resuscitation science review process: update

1. ILCOR resuscitation science review process:update New Orleans Nov 5th 2008Peter Morley

2. Conflict of interest disclosure • Commercial/industry • Evidence Evaluation Expert (ILCOR/AHA) • Potential intellectual conflicts • Deputy Chair Australian Resuscitation Council (ARC) • Chair ALS Committee ARC • ARC rep on International Liaison Committee on Resuscitation (ILCOR)

3. Purpose of C2010 process:Summary of science Worksheets = publishable systematic reviews

4. Plan of attack • Summary (worksheet) • Worksheet flow • Identification of questions • Questions related to interventions • Questions related to diagnosis/prognosis • Worksheet examples

6. Quality items for meta-analyses • Were specific objectives of the review stated (based on a specific clinical question in which patient, intervention, comparator, outcome (PICO) • Was study design defined? • Were selection criteria for stated for studies to be included (based on trial design and methodological quality)? • Were inclusive searches undertaken (using appropriately crafted search strategies)? • Were characteristics and methodological quality of each trial identified? • Were selection criteria applied and a log of excluded studies with reasons for exclusion reported?

7. Plan of attack • Summary • Worksheet flow • Identification of questions • Questions related to interventions • Questions related to diagnosis/prognosis • Worksheet examples

8. Internet sitemc.manuscriptcentral.com/ilcor

9. Worksheet flow for authors (1): • Author notified of worksheet question • Author registers COI • Author accesses instructions on internet • Author submits initial search strategy* • Author continues with worksheet • Author submits subsequent revisions of worksheet

10. Revision flow for authors (2): • Feedback provided • Staff (Tanya) • Taskforce co-chair (or delegate) • Worksheet expert • Evidence Evaluation Expert • Incomplete (feedback provided) • Complete for interim (need 2009 update) • Worksheet posted on intranet

11. Worksheet flow for authors (3): • Final 2009 worksheet submitted • incorporating 2009 search data • Incorporating feedback • task forces, WSE, E3 • Incomplete (feedback provided) • Complete • Worksheet posted on internet

12. Worksheet flow for authors (4): • Worksheet feedback from community: • Incomplete (feedback provided) • Complete • Worksheet presentations (2008/9/10) • Worksheet information incorporated into Consensus on Science document

13. C2010 Worksheet Flow Draft Author submits WS online Revise Preliminary review by office Worksheet Expert Review Revise WS Posted to the ExtraNet Complete Incomplete WS Posted to Internet Review/Sign off by Task Force Chairs Revise C2010 Conference CoSTR Writing CoSTR Review by Editorial Board CoSTR Publication

14. Plan of attack • Summary • Worksheet flow • Identification of questions • Questions related to interventions • Questions related to diagnosis/prognosis • Worksheet examples

15. Have we flagged all the questions that we need to answer for C2010??? Ongoing process!

16. Best signs to identify Number of initial breaths? CV ratio, rate? Signs of life? Energy levels & waveforms? Duration of CPR? Which airway? RR and Vt? Vasopressors? Timing!! Which signs? Alter management? Antiarrhythmics? Etc, etc? Ensure not omit Qs:major/contentious

17. Plan of attack • Summary • Worksheet flow • Identification of questions • Questions related to interventions • Questions related to diagnosis/prognosis • Worksheet examples

18. Flow chart for all LOEs Does the study directly address the question asked? Yes No LOE = 1 to 4 LOE = 5

19. Flow chart for intervention LOEs Does the study have its own control group? Yes No LOE = 1 to 3 LOE = 4

20. Flow chart for intervention LOEs Does the study use concurrent controls? Yes No LOE = 1 or 2 LOE = 3

21. Flow chart for intervention LOEs Does the study use true randomisation? Yes No LOE = 1 LOE = 2

22. Plan of attack • Summary • Worksheet flow • Identification of questions • Questions related to interventions • Questions related to diagnosis/prognosis • Worksheet examples

23. C2010 LOEs for Diagnostic Studies • LOE D1: Validating cohort studies (or meta-analyses), or validation of Clinical Decision Rule (CDR) • LOE D2: Exploratory cohort study (or meta-analyses), or derivation of CDR, or split-sample validation only • LOE D3: Diagnostic case control study • LOE D4: Study of diagnostic yield (no reference standard) • LOE D5: Studies not directly related to the specific patient/population (eg. different patient/population, animal models, mechanical models etc.)

24. Quality for diagnostic studies Consistent aim = minimize bias

25. Diagnostic cohort (studies of test accuracy) or case-control studies (D1, D2, or D3) • Was the diagnostic test evaluated in an appropriate spectrum of patients (like in those in whom it would be used in practice; spectrum bias”)? • Was there an independent, blind comparison (review bias) with a reference ("gold") standard of diagnosis? • Was the reference standard applied regardless of the test result (verification bias)?

26. Diagnostic studies without reference standard (LOE D4) (As for other LOEs with out controls) • Were outcomes measured in an objective way? • Were known confounders identified and appropriately controlled for? • Was follow-up of patients sufficiently long and complete?

27. Diagnosis outcomes

28. Diagnosis outcomes • Sensitivity • = those with the disease who were identified by the +ve test / all with disease • Specificity • = those without the disease who were identified by the -ve test / all without disease

29. Diagnosis outcomes • Likelihood ratio for a positive test • = probability +ve test in those with disease / probability +ve test in those without disease [= sens/(1-spec)] • Likelihood ratio for a negative test • = probability -ve test in those without disease / probability -ve test in those with disease [= (1-sens)/spec)]

31. C2010 LOEs for Prognostic Studies • LOE P1: Inception (prospective) cohort studies (or meta-analyses of inception cohort studies), or validation of Clinical Decision Rule (CDR) • LOE P2: Follow up of untreated control groups in RCTs (or meta-analyses of followup studies), or derivation of CDR, or validated on split-sample only • LOE P3: Retrospective cohort studies • LOE P4: Case series • LOE P5: Studies not directly related to the specific patient/population (eg. different patient/population, animal models, mechanical models etc.)

32. Quality for prognostic studies Consistent aim = minimize bias

33. New versus updated worksheet • Will be determined by task-force • Requirements for searching will be different (2004 onwards) • Need to refer to articles cited in previous worksheet • Summary from C2005 can be pasted and designated

34. Worksheet: 9 separate sections • Basic demographics • Clinical Question • Declaration regarding conflict of interest • Search strategy and results • Summary of the evidence • Reviewer’s final comments • Conclusion • Acknowledgements • Citation list

35. Search strategy • Two of the most important factors that are related to the final quality of a systematic literature review are: • a clearly stated search strategy • a clearly defined inclusion and exclusion criteria for the final evaluation phase • = “methodology section” of the worksheet based systematic review

37. Deciding Search Terms • first select the most unique concept • Combining searches • In practice, often there is no C (comparison), and you may be interested in all O (outcomes) • Starting with a simple combination of the Population and Intervention is often the best approach: P and I.

38. Combining searches • Use Boolean Operators • “AND” restricts searches: to be retrieved a reference must be indexed with every term searched. The more terms that are combined with ‘and’, the smaller the number of references retrieved. • “OR” extends searches: to be retrieved a reference just needs one of the terms searched. The more terms you ‘or’, the greater the number of references retrieved.

39. Medical Subject Headings (MeSH) • The National Library of Medicine uses a controlled vocabulary, referred to as Medical Subject Headings (MeSH) • cross-reference every single abstract indexed on Medline. The use of a controlled vocabulary provides • a consistent way to retrieve information that may use different expressions to describe similar concepts • To ensure that this controlled vocabulary is reliably applied, when a paper is submitted for indexing on Medline, a professional nosologist assesses the paper and indexes it using the appropriate MeSH terms.

40. Refining your search terms • use the Details button to evaluate your search terms • use the MeSH database to refine your MeSH terms • use the display citation feature to find MeSH categories

41. “three step process” • the generation of a series of focused clinical questions to identify potential search terms • The use of PubMed Clinical Query search filters • The use of: • Details feature • MeSH Database • Display Citation feature to select and refine appropriate MeSH terms.

42. Search strategies • 1. exp Heart Arrest/ • 2. exp Death, Sudden/ • 3. cardiac arrest.tw. • 4. heart arrest.tw. • 5. cardiopulmonary arrest.tw. • 6. sudden cardiac death$.tw. • 7. sudden death$.tw. • 8. or/1-7 • 9. exp Cardiopulmonary Resuscitation/ • To 67 steps

43. Succinct example • "infusions, intravenous"[MeSH Terms] AND ("heart arrest"[MeSH Terms] OR "cardiopulmonary resuscitation"[MeSH Terms]) • ("cardiac only" OR "compression only" OR "compressions only" OR "hands only") AND ("heart arrest"[MeSH Terms] OR "cardiopulmonary resuscitation"[MeSH Terms])