1 / 24

Supporting systematic reviews for STEM researchers & educators

Supporting systematic reviews for STEM researchers & educators. Texas STEM Librarians’ Conference July 17, 2014. Margaret J. Foster, MS, MPH, AHIP Assistant Professor Systematic Reviews and Research Coordinator Texas A&M University Libraries. Overview. What are systematic reviews?

lali
Download Presentation

Supporting systematic reviews for STEM researchers & educators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Supporting systematic reviews for STEM researchers & educators Texas STEM Librarians’ Conference July 17, 2014 Margaret J. Foster, MS, MPH, AHIP Assistant Professor Systematic Reviews and Research Coordinator Texas A&M University Libraries

  2. Overview • What are systematic reviews? • Role of librarians in systematic reviews • Center for Systematic Reviews at Texas A&M University Libraries • STEM Systematic Reviews

  3. “The notion of systematic review – looking at the totality of evidence – is quietly one of the most important innovations in medicine over the past 30 years.” Dr. Ben Goldacre, award-winning Bad Science columnist and medical doctor, forward in Testing Treatments

  4. What are systematic reviews? A systematic review “attempts • to collate all empirical evidence that fits pre-specified eligibility criteria • in order to answer a specific research question. • It uses explicit, systematic methods that are selected with a view to minimizing bias, • thus providing more reliable findings from which conclusions can be drawn and decisions made.”1 1 Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org., Section 1.2.2

  5. Texas STEM Librarians’Conference • Slide#5 Comparing review types Scientific approach to a review article Criteria determined at outset Comprehensive search Explicit methods of appraisal and synthesis Depend on authors’ inclination (bias) Author selects any criteria Search any databases Methods not usually specified Can’t replicate review Narrative Reviews Systematic Reviews Subjective Objective

  6. citation Steps of systematic review methods The process should be: • Transparent: record & report all methods • Follow standards and evidence based practices • Minimize bias

  7. Specific types of reviews Types vary by • Specificity of the question • Comprehensiveness of search and inclusion • Depth assessment and coding • Type of synthesis • Rapid review: narrow, quick search and assessment of very specific question • Scoping review: assessment of potential size/scope of available literature • Integrative review: includes qualitative/ quantitative/ theory • Meta-analysis: quantitative synthesis • Umbrella review: review of systematic reviews Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x

  8. What makes this search different?? • It will be evaluated by many • All articles the search retrieved must be labeled • Search one database at a time • Sensitivity is more important than specificity • Bias during the search is a one of the biggest threats to the review

  9. Publication bias When due to nature & direction of research findings a situation occurs making it more difficult to locate studies Rothstein, D. H. R., Sutton, D. A. J., & Borenstein, D. M. (2006). Publication Bias in Meta-Analysis. Publication Bias in Meta-Analysis(pp. 1-7) doi:10.1002/0470870168.ch1

  10. Identification Records retrieved from database searching (n=3000 ) Records retrieved from additional sources (n=1500 ) PRISMA Total retrieved records (n=4500 ) Duplicates removed (n= 2000 ) Preferred Review Items for Systematic reviews and Meta Analysis flowchart Screening Records screened by title (n=3500 ) Records excluded (n=2500 ) Records screened by abstract (n=1000 ) Records excluded (n= 700) 300 wrong population 400 wrong intervention Eligibility Records excluded (n=250 ) 100 wrong population 100 wrong intervention 50 wrong education issue Records screened by full text (n=300) Included Records included in qualitative synthesis (n=500 ) Records excluded (n= 25 ) 12 wrong outcome 13 lack of follow up Records included in quantitative synthesis (n= 25) See full PRIMSA statement at http://www.prisma-statement.org

  11. Role of Librarian in SRs McGowan, J., & Sampson, M. (2005). Systematic reviews need systematic searchers. Journal of the Medical Library Association : JMLA, 93(1), 74-80.

  12. Team- roles in steps Adapted from: McGowan, J., & Sampson, M. (2005). Systematic reviews need systematic searchers. Journal of the Medical Library Association : JMLA, 93(1), 74-80.

  13. Standards Institute of Medicine of the National Academies (2011) has two standards specifically referring to librarians: • 3.1.1 Work with a librarian or other information specialist trained in performing systematic reviews to plan the search strategy • 3.1.3 Use an independent librarian or other information specialist to peer review the search strategy 27 items in the PRISMA check list… • 7. “information sources,” “describe all information sources…in the search and date last searched.” • 8. “Search,” “present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.” Standards for Systematic Reviews from Institute of Medicine of the National Academies http://www.iom.edu/~/media/Files/Report%20Files/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards%20for%20Systematic%20Review%202010%20Insert.pdf PRIMSA statement at http://www.prisma-statement.org

  14. Authorship (Vancouver statement) Authorship credit should be based on: Authors should meet conditions 1, 2, and 3 substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data drafting the article or revising it critically for important intellectual content final approval of the version to be published. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. JAMA. 1997. Mar 19; 277(11):927.–34.

  15. Center for Systematic Reviews Who can use the service? all Texas A&M University students, faculty, and staff What is offered? • Consultations • Class presentations • Webinars • Workshops Majority of requests from: • College of Medicine • School of Public Health • College of Veterinary Medicine • College of Education • Department of Health & Kinesiology • Educational Psychology • College of Agriculture

  16. Consultations • Initial consultations • Establish that question is sound • Set up criteria • Research plan • Locate relevant reviews • Discuss type of consultation • Follow up consultations • Continue search • Demonstrate steps/ provide examples

  17. Evaluating the service Annual survey: Researcher/review demographics State of review/ level of understanding of SRs before & after consultations Consultations Tools offered by services Future services Comments

  18. Levels of consultation

  19. Reviews in STEM education • Effectiveness of educational interventions • Test theories • Identify gaps in current research • Provide organized evidence for decision making • Help in planning new interventions

  20. SR Organizations in education

  21. Overview of SRs in Eng. Education • Introduction • Systematic reviews in other disciplines • Purposes and goals of reviews • Steps in conducting the review • Deciding to do a systematic review • Identifying scope and research questions • Defining inclusion criteria • Finding and cataloging sources • Critique and appraisal • Synthesis • Limitations, validity, and reliability concerns • Systematic identification of reviews Borrego, M., Froyd, J., and Foster, M. (2014) Systematic Literature Reviews in Engineering Education and Other Developing Interdisciplinary Fields. Journal of Engineering Education 103(1): 45-76. DOI: 10.1002/jee.20038

  22. Number of Reviews published

  23. Critical Analysis of SRs • We started with 473 articles, resulting in 49 included reviews: 17 MAs, 32 SRs • Used Campbell Collaboration checklist to evaluate the article • Found that authors’ of reviews did not… • adequately describe inclusion criteria • conduct appropriate comprehensive search • independently screen or assess articles • include flowchart of reasons articles were excluded • appropriately address reviews limitations Campbell Collaboration. (2012). Education coordinating group systematic review checklist. from http://www.campbellcollaboration.org/artman2/uploads/1/ECG_Protocol-Review_Checklist_1108.pdf

  24. Conclusion • Systematic reviews are becoming more common in disciplines outside of medicine • To conduct SR searches appropriately requires some training • SRs provide opportunity for collaboration with researchers

More Related