Systematic Reviews: Database Selection, Search Strategies & Reference Management

1. Systematic Reviews:Database Selection, Search Strategies & Reference Management Christopher Stave, MLS Instructional Program Coordinator Lane Medical Library & Knowledge Management Center

2. Overview • Definition of a systematic review • Elements of systematic review • Guides and standards • Software tools • Developing a search strategy • Database selection and “grey literature” • Fine-tuning your search strategies • Documenting your searches • Systematic review search flow-chart • Tools for managing references/PDFs/bibliographies

3. What is a systematic review? • “A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review.” • “Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies.” • “Meta-analysis refers to the use of statistical techniques in a systematic review to integrate the results of included studies.” Prisma: “Preferred Reporting Items for Systematic Reviews and Meta-Analyses“ Prisma Statement: http://www.prisma-statement.org/statement.html Accessed 9 Aug 2010.

4. Systematic vs narrative reviews • Narrative reviews are summaries of research • Generally lack explicit descriptions of systematic methods • Evidence is often incomplete • Relevance and validity of studies often not explicit • Tend to provide a wider view of a topic “The Cochrane Collaboration: What is a systematic review?” http://www.cochranemsk.org/cochrane/review/default.asp?s=1. Accessed August 10, 2010

5. Systematic reviews require… • Comprehensive and well-formulated Searches • Careful Assessment • Synthesis of relevant studies

6. Elements of a systematic review • Clearly defined question • Comprehensive search • Explicit inclusion criteria • Assessments of methodological quality • Synthesis of data • Summary of results

7. The raw material… • Each included study is considered a “unit of analysis” with eligibility criteria determining inclusion

8. Guides to formulating studies and determining quality • PRISMA (formerly QUORUM) • Preferred Reporting Items for Systematic Reviews and Meta-Analyses • CONSORT • Consolidated Standards of Reporting Trials (RCTs) • GRADE • Grading of Recommendations Assessment, Development and Evaluation (sys reviews, practice guidelines) • JADAD Scale • Assessment of methodological rigor of a clinical trial • STROBE • Strengthening the Reporting of Observational studies in Epidemiology • EQUATOR • Enhancing the Quality and Transparency Of Health Research. Involved in monitoring guidelines for research

9. Software • Reference/PDF/bibliography tools • EndNote • Zotero • Mendeley • Papers • Systematic review tools • RevMan 5 • GradePro • Comprehensive Meta-Analysis (CMA)

10. Developing an effective search for a systematic review • Effective search strategies depend on a clearly articulated research question • Search strategies should take advantage of the unique structure and search utilities of each database • Identifying variant terminology (synonyms) for specific concepts will increase retrieval

11. Developing an effective search for a systematic review • Be SURE to consult with a research librarian to help you: • select the appropriate databases for your search • develop search strategies that strike an appropriate balance between comprehensiveness (recall) and “exactness” (precision) • determine which reference/PDF/bibliography management tool(s) are best for your project • Establish a realistic timeline with the librarian: systematic reviews are real projects not just quickie PubMed searches

12. Selecting a database • Most disciplines have specialized databases • Lane provides links to a subset of all Stanford databases, focusing primarily on biomedicine • A more comprehensive list can be found on the Stanford University database website

13. “Grey Literature” • Unpublished or hard-to-find studies, i.e., “grey literature” presents a challenge to the searcher • Depending on the topic, the searcher may have to search: • Conference or society websites for meeting abstracts • University or corporate trial registries • Databases specializing in governmental research, e.g., NTIS Database and RePORT • Databases of clinical trials: Cochrane Controlled Trials Register, or Clinicaltrials.gov • Google or Google Scholar

14. Database selection example:“Is gardening good for you?” • Agriculture • CAB Abstracts • Biomedicine: • PubMed • Cochrane Library • CINAHL (nursing and allied health) • Psycinfo • Toxnet • Economics/Business • ABI/Inform

15. Database selection example:“Is gardening good for you?” • Social Sciences: • ERIC (education) • Sociological Abstracts • Multi-disciplinary databases • SCOPUS • Web of Science • Grey literature • NTIS, Cochrane CTR, Clinicaltrials.gov • Search engines • Google/Google Scholar

16. Fine-tuning your search strategies • Search strategy development is iterative • Storing and sharing searches (e.g., with PubMed’s “My NCBI”) allows for collaborative editing • Use a set of previously identified key articles, and check to see if they’re retrieved by the initial search strategies: if not, the search may need to be edited, or other databases may need to be included • Once the strategies have been optimized, most databases provide an auto-alert feature that automatically searches/emails references at specific frequencies (week/month)

17. Documenting search strategies • List databases and vendor (if relevant); e.g., MEDLINE/OVID Technologies • Note date-range searched and dates of last search, number of references retrieved, and exact search strategy(ies) • Note limits by topic/language/publication-type limits (e.g., human/Eng/RCTs) • List individuals or organizations contacted • List “gray literature” sources • Document other search strategies (e.g., scanning bibliographies of articles) Database/Vendor: Medline/OVID DATE: 1950s –2009/last searched February 29, 2009 LANGUAGE: English PUBLICATION TYPES: Randomized controlled trials Total: xxx Strategy: Exact search strategy. Should be replicable Database/Vendor:American Heart Association, Abstracts From Scientific Sessions 2009, search of AHA Abstracts Online DATE: 2009/last searched February 29, 2009 LANGUAGE: English Total: xxx Strategy: Exact search strategy. Should be replicable Based on: “Reporting the search process in the review” Cochrane Handbook for Systematic Reviews of Interventions. Version 5.02, Sep 2009.

18. Search flow-chart • Total Identified (n=1200) • Database 1: 1000 • Database 2: 40 • Database 3: 160 Two Reviewers Articles requiring title/abstract review (n=1200) • Excluded (n=1000) • Reason 1: 800 • Reason 2: 200 Articles requiring full-text review (n=200) Two Reviewers • Excluded (n=180) • Reason 1: 100 • Reason 2: 80 Two Reviewers Data Extraction (n=20) Based on the 2009 PRISMA flow diagram

19. Managing References/PDFs/Bibliographies: Some considerations… • Collaboration and sharing of PDFs/references • Stanford-affiliated? • Non-Stanford affiliated? • Web-based vs desktop • PDF downloading and linking • Creating groups and adding searchable fields • Exportability • What tools are available…?

21. Example: “Does Echinacea prevent and/or treat the common cold?” • Identify basic search parameters and limits (these may be changed depending on retrieval and/or project resources) • English • Randomized controlled trials • No date restriction • Select databases, registries, search engines, e.g., • PubMed, Cochrane Library, SCOPUS, CINAHL, EMBASE*, Clinicaltrials.gov • Identify variant terminology • Save searches *Much of the content of EMBASE is included in SCOPUS

22. Example: “Does Echinacea prevent and/or treat the common cold?” • Share “My NCBI” searches with team members • Did ALL previously identified relevant articles (if any) appear in the search? No? Find out why! • Did articles identified in bibliographies appear in the search? No? Find out why! • Do team members have suggestions for expanding/restricting search based on additional terms? • Once the search has been vetted, a “My NCBI” auto-alert should be set up • Choose an appropriate reference/PDF management application, e.g. EndNote, and import the retrieved references from PubMed

23. Example: “Does Echinacea prevent and/or treat the common cold?” • Adapt the PubMed searches to other databases, registries and search engines • Import references and delete duplicates • Keep track of search dates, ranges, number of references retrieved, and strategies

24. LIVE DEMO! “Does Echinacea prevent and/or treat the common cold?” • PubMed search • My NCBI: commoncoldech/commoncoldech • Search is vetted and approved • EndNote • Download references into EndNote • Create “group sets” and ”groups” • Create additional fields and change the field display • Access and download PDFs for relevant articles (configure EndNote with http://sfx.stanford.edu/local) • SCOPUS search • Google Scholar search • Using Zotero to capture references

25. Echinacea and Common Cold: Searching Flow-chart • Total Identified (n=1300) • PubMed: 1000 • SCOPUS: 200 • Google Scholar: 100 Two Reviewers Articles requiring title/abstract review (n=1300) • Excluded (n=1000) • Not RCTs: 800 • Not English: 200 Articles requiring full-text review (n=300) Two Reviewers • Excluded (n=200) • No incidence/duration: 100 • Not prevention or tx: 100 Two Reviewers Data Extraction (n=100) Based on the 2009 PRISMA flow diagram

26. Echinacea and Common Cold: Search strategy documentation • List databases and vendor (if relevant); e.g., MEDLINE/OVID Technologies • Note date range searched and dates of last search • Note language/publication-type limits • Include total references retrieved • Include exact search strategy Database/Vendor: PubMed/NCBI DATE: 1950s –2010/last searched August 10, 2010 LANGUAGE: English PUBLICATION TYPES: Randomized controlled trials Total: 1000 Strategy: (("Echinacea"[Mesh] AND "Common Cold"[Mesh]) AND (Randomized Controlled Trial[ptyp] AND English[lang])) OR (("common cold" OR rhinovirus*) AND echinacea AND (random* OR blind* OR control*)) Based on: “Reporting the search process in the review” Cochrane Handbook for Systematic Reviews of Interventions. Version 5.02, Sep 2009.

27. Additional resources • Cochrane Handbook for Systematic Reviews of Interventions • Articles -- Systematic Review Example 1: Khazeni N, Bravata DM, Holty JE, Uyeki TM, Stave CD, Gould MK. Systematic review: safety and efficacy of extended-duration antiviral chemoprophylaxis against pandemic and seasonal influenza. Ann Intern Med. 2009 Oct 6;151(7):464-73. PMID: 19652173. • Articles -- Systematic Review Example 2: Caughey AB, Sundaram V, Kaimal AJ, Gienger A, Cheng YW, McDonald KM, Shaffer BL, Owens DK, Bravata DM. Systematic review: elective induction of labor versus expectant management of pregnancy. Ann Intern Med. 2009 Aug 18;151(4):252-63, W53-63. PMID: 19687492.

28. Questions?

29. Contact info Christopher Stave, MLS Lane Medical Library Stanford University Medical Center 650 725-4580 cstave@stanford.edu