Grey Matters! Finding Grey Literature

1. Grey Matters! Finding Grey Literature Amanda Hodgson, MLIS Sarah Normandin, BSc, MLIS CE Course May 26, 2008 CHLA 2008

2. What is Grey Literature? • Grey • Gray • In-house • Unpublished • Fugitive • Deep Web

3. What is Grey Literature? International Conference on Grey Literature. "Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body." (Luxembourg, 1997 - Expanded in New York, 2004) http://www.greynet.org/greynethome.html Document types in grey literature: http://www.greynet.org/greysourceindex.html • Source: GreyNet: grey literature network service. Amsterdam: GreyNet; 2008.

4. What is Grey Literature? • How are grey literature definitions changing…or going grey? • What is grey literature searching? • Remember, the search is greater than the definition!

5. Benefits of Grey Literature • Current • Often free • Relevant and unique • Information on non-mainstream topics or aspects • Aids in identifying terms for search strategies

6. Publication Bias • Necessary for certain types of research, like systematic reviews --- • “…. published trials tend to be larger and show an overall greater treatment effect than grey trials. This has important implications for reviewers who need to ensure they identify grey trials, in order to minimise the risk of introducing bias into their review.” • Source: Hopewell S, McDonald S, Clarke MJ, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev 2007;(2).

7. Benefits of Searching for Grey literature • “Unpublished trials contribute about 20% of the weight in individual meta-analysis.” • Source: Krishnan RR. Evidence-based practice: how to read what you read. Psychopharmacology Bulletin 2004;37(4):88-91. [our emphasis]

8. Challenges of Grey Literature • Hard to find • Much content isn’t web-published • Rarely indexed, or minimally indexed • Basic bibliographic information may be unavailable • Older reports may not be archived • Volume of material can be overwhelming and time-consuming

9. Challenges of Grey Literature • Evidence provided by grey literature may be criticized.

10. Starting your Grey Literature Search • Who is the recipient of the search results? • What is the resulting information product? • What are your search parameters, and in what ways must they be either flexible or more rigorously applied?

11. CADTH Search Approach • Scope definition (PICO) and search terms • 2. Comprehensiveliterature search • Database search for commercially published literature (e.g. Ovid databases, PubMed) • (b) “Grey Literature” component • Often started at Step 1 to unearth search terms • CADTH Grey Literature Checklists

12. (b) Grey Literature Checklists • CADTH-standard lists • customized for each report type • PLUS included in the “grey literature” package: • May include both commercially published literature as well as grey literature

13. CADTH Grey Literature Checklists • New! Single compiled list available online: • Grey Matters: a practical tool for evidence-based searching • http://cadth.ca/ • About CADTH > CADTH Products (scroll down)

14. Tips for Saving Time • Is general information or a comprehensive search required? • Scope out major systematic reviews (e.g. Cochrane reviews) and HTAs first • Search strategy and keyword ideas • Have a well-developed strategy in advance • Reduces backtracking • Use a checklist for tracking sources • Corollary: don’t spend too much time tracking! • Record minimal details to fully cite later

15. Exercises: Grey Literature Sources • Health Technology Assessment (HTA) agencies • Health Economics • Clinical Practice Guidelines • Drug/Devices Approval • Advisories/Warnings • Drug Class Reviews • Clinical Trials • Databases • Statistics • Open Access Journals • Additional Grey Literature Types and Sources

16. Class Exercises: Case Studies • PICO Model: defining the research question Population (Patient, Population, Problem) Intervention Comparator Outcome

17. PICO Example • Question: Conduct a grey literature search on combination long-acting beta-2 agonists (LABA) plus corticosteroids inhalers for asthma, compared to separate medication treatments. • Asthma patients • Long-acting beta-2 agonists (LABA) plus inhaled corticosteroids (ICS) administered by inhaler • LABA alone • ICS alone • other asthma medications/inhalers • Better control of asthma symptoms; reduced airflow obstruction P I C O

18. Health Technology Assessments (HTAs) • Highest quality evidence comes from: • Systematic reviews (SR) • Meta-analyses (MA) • Health technology assessments (HTA)

19. Systematic Review • Review of studies in which evidence has been systematically searched for, studied, assessed, and summarised according to predetermined criteria. Definitions based on NLM, Cochrane Glossary, INAHTA, CADTH definitions.

20. Meta-analysis • Use of statistical techniques for combining results from different studies to obtain a quantitative estimate of the overall effect of a particular intervention, potentially producing a stronger conclusion than any individual study

21. Health Technology Assessment • Multidisciplinary and systematic policy analysis, that studies the clinical, social, ethical, and economic implications of a drug, medical system, or health technology. • Primary purpose is to inform health policy and clinical practice decision makers.

22. HTA Highlights - Canada • CADTH • Canadian Agency for Drugs and Technologies in Health • http://cadth.ca/ • AETMIS • Agence d’évaluation des technologies et des modes d’intervention en santé • http://www.aetmis.gouv.qc.ca

23. HTA Highlights - International • HTAi Vortal • Health Technology Assessment International • http://www.htai.org/ • NCCHTA • National Coordinating Centre for Health Technology Assessment • http://www.ncchta.org/

24. Health Economics • Focus is on choices, value judgements with regard to health care decision making • Direct/indirect costs • Statistics, incidence/prevalence • Socioeconomic • Quality of life

25. Health Economics • Economic evaluations • Cost-minimization analyses • Cost-effectiveness analyses • Cost-utility analyses • Cost-benefit analyses • At CADTH, we may be conducting searches for economic evaluations.

26. Health Economics • Highlights • Centre for Health Economics and Policy Analysis (CHEPA) • http://www.chepa.org • EURONHEED(European Network of Health Economic Evaluation Databases) • http://infodoc.inserm.fr/euronheed/Publication.nsf

27. Clinical Practice Guidelines • A search for clinical practice guidelines may extend to association websites or organizations dedicated to a particular disease or patient group, depending on the requirements of the search. • Guidelines • Protocols • Positions statements • Patient information sheets and resources

28. Clinical Practice Guidelines • Highlights • Guidelines & Protocols Advisory Committee, British Columbia Ministry of Health • http://www.hlth.gov.bc.ca/msp/protoguides/index.html • National Guideline Clearinghouse • http://www.guideline.gov

29. Drug and Devices Approvals • Regulatory approvals must be granted by government bodies for drugs and health devices. Information on the expansion of patient groups for previously approved drugs or withdrawals of medicines/devices from the market are often available.

30. Drug and Devices Approvals • Highlights • Medical Devices Active License Listing (Canada) • http://www.mdall.ca • Center for Drug Evaluation and Research (CDER) • http://www.accessdata.fda.gov/scripts/cder/drugsatfda/

31. Advisories/Warnings • Medical advisories and warnings are reported by governments internationally. • Adverse effect/events • Safety concerns (devices and medical procedures) • Withdrawals from the market

32. Advisories/Warnings • Highlights • Health Canada • MedEffect: Canadian Adverse Reaction Newsletter, Health Canada • http://www.hc-sc.gc.ca/hpfb-dgpsa/tpd-dpt/index_adverse_newsletter_e.html • European Agency for the Evaluation of Medicinal Products (EMEA) • Product safety announcements • http://www.emea.europa.eu/htms/human/drugalert/drugalert.htm

33. Drug Class Reviews • Highlights • DERP: The purpose of DERP reports is to make available information regarding the comparative effectiveness and safety profile of different drugs within pharmaceutical classes. • Drug Effectiveness Review Project (DERP), Oregon Health & Science University • http://www.ohsu.edu/drugeffectiveness/reports/index.cfm

34. Clinical Trials • Valuable information on the safety and efficacy of new drugs and therapies as well as new uses for known treatments • Information on studies that may not be published yet • Tip: Check the product manufacturer’s website for trial lists • ClinicalTrials.gov (U.S. National Institutes of Health) • http://clinicaltrials.gov/ • Current Controlled Trials • http://www.controlled-trials.com/

35. Databases • Retrieve grey literature in addition to traditional commercially published literature • NHS CRD • Centre for Reviews & Dissemination, University of York, UK • http://www.crd.york.ac.uk/crdweb/ • Cochrane Library (free full text in NB, NS, NU, SK, YK, NWT) http://www.thecochranelibrary.com/ • TRIP Database http://www.tripdatabase.com/

36. Statistics • Statistics on incidence and prevalence of specific conditions, population information • Consult association websites for statistics on specific diseases not covered on major statistical websites • Statistics Canada http://www.statscan.ca/ • Public Health Agency of Canada (PHAC) • http://www.phac-aspc.gc.ca/

37. Open Access Journals • Journals where all content is freely available, no embargo period • Often peer reviewed • PubMed Central http://www.pubmedcentral.com • Directory of Open Access Journals http://www.doaj.org

38. Additional Grey Literature Sources • Online catalogues • Amicus, NLM, British Library (BL) • Company Directories and Patents • Industry Canada databases: http://www.ic.gc.ca/ Patents database: http://www.collectionscanada.gc.ca/innovations/023020-5000-e.html • Blogs/Discussion lists • CanMedLib • Medical writers • Patient advocates

39. Conferences/Congresses/Symposia • Manufacturer/Association websites • Input from experts/researchers • Online conference planners, conference reporting • http://www.medscape.com • http://www.doctorsreview.com/ • http://www.biomedcentral.com/browse/abstracts/ • Database records • ex. Biosis • Internet search • (disease OR drug) AND (meeting* OR abstract* OR poster* OR conference* OR proceeding* OR poster*)

40. Managing and Presenting Grey Literature Findings • Presenting results • Difficult citations • Citing Medicine: the NLM style guide for authors, editors, and publishers. 2nd edition. Bethesda (MD): U.S. National Library of Medicine; 2007. Available: http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=citmed.TOC&depth=2 • Citation management • ProCite, Reference Manager, EndNote, etc. • Keeping grey literature results separate vs. integrating with database search results

41. Alerts and Awareness Tools • Limit searches by date range • Google Advanced – last 24h, week, month(s), year • E-mail alerts & RSS feeds • Set up alerts in databases such as PubMed (MyNCBI), Cochrane, CRD • RSS feeds: PubMed, MedScape, ClinicalTrials.gov, journals

42. Alerts & Awareness Tools • Webpage change detection services • http://watchthatpage.com/ • http://trackengine.com/

43. Closing Remarks • Tips for your grey literature search! Have a well-developed strategy before you start Re-negotiate PICO & parameters as you search Keep it simple: paste title and link into a “good copy” document as you proceed Grey literature searching is key for finding certain types of information and reducing publication bias

44. References of Interest - Canadian • Grey matters: a practical tool for evidence-based searching. Ottawa: Canadian Agency for Drugs and Technologies in Health;2008. http://cadth.ca/media/pdf/Grey-Matters_A-Practical-Search-Tool-for-Evidence-Based-Medicine.doc • Giustini D, Thompson D. “Finding the Hard to Finds”: Annotated bibliography (Appendix E). Vancouver: University of British Columbia; 2006. http://weblogs.elearning.ubc.ca/googlescholar/greylit_manual_May11.pdf • IHE report: health technology assessment on the net: a guide to Internet sources of information. Edmonton: Institute of Health Economics; 2007. http://www.ihe.ca/documents/ihe/publications/reports/IHE_Report_Health_Technology_Assessment_on_the_Net_Jun_2007.pdf

45. References of Interest - International • GreyNet (Grey Literature Network Service). http://www.greynet.org/ • New York Academy of Medicine. Grey Literature Report. http://www.nyam.org/library/pages/grey_literature_report# • Helmer D. Chapter 10: Grey literature. In: Auston I, Topfer L, eds. Etext on health technology assessment (HTA) information resources. Bethesda (MD): National Information Center of Health Services Research and Health Care Technology (NICHSR); 2006. http://www.nlm.nih.gov/archive////2060905/nichsr/ehta/chapter10.html

46. Questions?