Methodological Issues in Systematic Reviews: Formulating Questions

1. Methodological Issues in Systematic Reviews: Formulating Questions Thomas A Trikalinos, MD, PhD Co-Director, Tufts Evidence-based Practice Center Author has no actual or potential conflicts of interest in relation to this activity

2. Asking the right question

3. We will cover • PICO(TS) approach to generate answerable research questions • Analytic framework to identify potential questions • Other considerations in formulating questions

4. Systematic Review/Meta-Analysis

5. Systematic review protocol • Well focused study question [ PICO(TS) ] • Identification of studies (studies design, source, search strategy) • Eligibility criteria (study, patient, and disease characteristics, treatments, outcomes) • Data extraction (definition of outcomes, quality assessment) • Data summary and analysis (outcomes used, intention to treat)

6. PICO(TS) approach to formulating answerable research questions • Population (participants) • Intervention • Comparator • Outcome • Time (duration) • Study design (experimental / observational, N)

7. Poorly formulated question:What drugs should be used to treat patients with hypertension?

8. Example: The Well Formulated Question

9. Diagnostic test research question- test performance - Condition of interest Setting population Outcomes Intervention What is the diagnostic accuracy of sestamibi scan to detect ACI in patients in the ED? Adults with symptoms suggestive of acute cardiac ischemia and non-diagnostic ECG seen in the ED sensitivity specificity • - AMI • unstable angina technetium-99m sestamibi scan Comparator

10. Diagnostic test research question- clinical impact - Condition of interest Setting population Outcomes Intervention What is the clinical impact of sestamibi scan to detect ACI in patients in the ED? Adults with symptoms suggestive of acute cardiac ischemia and non-diagnostic ECG seen in the ED False negative discharges from the ED False positive admits Deaths • - AMI • unstable angina technetium-99m sestamibi scan Comparator

11. We will cover PICO(TS) approach to generate answerable research questions Analytic framework to identify potential questions Other considerations in formulating questions

12. Approach to (de)constructing research question • Question of interest is often over-arching and not directly addressable • Ex. How to manage hypercholesterolemia? • Need to break it down into answerable subunits • Multiple questions frequently needed • Final answers (recommendations) typically involve additional information, balancing the benefits and harms, feasibility, costs, etc

13. Simplified Evidence Model Linking Biological Effects (intermediate) and Health Outcomes

14. USPSTF generic analytic framework for screening topics

15. Analytic framework for omega-3 FA intake and CVD

16. We will cover PICO(TS) approach to generate answerable research questions Analytic framework to identify potential questions Other considerations in formulating questions

17. Problems defining interventions and comparators Exercise training AND OPTIONALLY: “Conventional” care Inspiratory muscle training Education Vs Breathing exercises Psychosocial/ behavioral interventions Pulmonary rehabilitation vs conventional care AHRQ Evidence Report/Technology Assessment Pulmonary rehabilitation for COPD, 2007; http://www.ahrq.gov

18. OK, but there are things that are self-evident, no?

19. Would You Say You “Had Sex” If… ? JAMA 1999;281:275-7

20. Percentage Indicating “Had Sex” JAMA 1999;281:275-7

21. Problems in coming up with questions • Sometimes you don’t know what should be the question because there is not yet agreement among investigators about the meaning • Pre-emptive analgesia: different investigators have different ideas about what it is • Solution: Rather than pick an arbitrary definition, try to group what has been studied into several similar categories and address them, this will provide evidence of the spectrum of mechanisms • Question: What is the effect of timing of analgesics on post-op requirements of pain medication and patient outcomes?

22. Problems with diagnosis • varying definitions (lack of an adequate reference standard) • different levels of rigor • tradeoffs between rigor and real world practice (sinus puncture to diagnose sinusitis vs. actual clinical practice: efficacy vs. effectiveness)

23. Sinus Trouble

24. Possible definitions of acute bacterial sinusitis and their effect

25. Management of Clinically Inapparent Adrenal Mass(Adrenal Incidentaloma):The Evidence Report NIH State-of-the-Science ConferenceFebruary 4, 2002

26. Incidentaloma is NOT a disease The prevalence of this phenomenon and the pathologies and their likelihood will depend on: • Patients with known cancers or not • Presenting signs and symptoms • Patient’s age and sex • Biochemical activity findings • Initial diagnostic test used (CT, US, MRI) • Size of the mass (lower limit)

27. Adenoma Adrenocortical carcinoma Angiomyelolipoma Cyst Epithelial carcinoma Hyperplasia Hematoma Lymphoma Metastases Myelolipoma Neurinoma Pheochromocytoma Regenerative hepatic nodule Renal angiomyolipoma Retroperitoneal fibrosis . . . more What are the pathologies reported among adrenal incidentaloma?

28. Some possible definitions of incidentaloma

29. Issues in formulating a question • Narrow versus broad (for individuals/ subgroups or entire population) • Clinically meaningful and useful (based on sound biological and epidemiological principles) • Very broadly defined questions may be criticized for mixing apples and oranges • Very narrowly focused questions have limited generalizability and sometimes may lead to biased interpretations / conclusions

30. Benefits of getting the questions right at the start • Improves clarity of the problem • Facilitates subsequent steps of the systematic review • Helps select right kind of studies for review • Anticipates issues, resource needs • Improves efficiency, reduces needs to revisit questions / re-review papers, minimizes wasted efforts

31. Summary • PICO(TS) method is a very useful technique to formulate answerable research questions • Analytic framework can be useful to visualize and think about related issues • Defining criteria • Biologically meaningful • Question and criteria guide the entire systematic review process