1 / 15

Penny Whiting, Marie Westwood , Roger Harbord, Margaret Burke, Jonathan Sterne, Julie Glanville

Can diagnostic filters offer similar sensitivity and a reduced NNR compared to searches based on index test and target condition?. Penny Whiting, Marie Westwood , Roger Harbord, Margaret Burke, Jonathan Sterne, Julie Glanville. Background.

meghan
Download Presentation

Penny Whiting, Marie Westwood , Roger Harbord, Margaret Burke, Jonathan Sterne, Julie Glanville

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Can diagnostic filters offer similar sensitivity and a reduced NNR compared to searches based on index test and target condition? Penny Whiting, Marie Westwood, Roger Harbord, Margaret Burke, Jonathan Sterne, Julie Glanville

  2. Background • Previous studies have shown that filters for diagnostic accuracy studies miss relevant studies • We have previously shown that that even searches designed to be very sensitive miss between 3 and 17% of studies indexed in MEDLINE • Therefore the use of a diagnostic filter may not be as problematic as previously reported

  3. Objective • To compare the number of relevant studies identified by searches of MEDLINE based on index test(s) and target condition(s) (“sensitive searches”), with the number of relevant studies identified by the same searches combined with a diagnostic search filter (“restricted searches”)

  4. Methods • Data: • 8 diagnostic reviews • CRD York and Bristol group reviews • Similar search approaches • No filters used on original searches • variety of topics, e.g. • Haematuria, PAD, bacterial vaginosis • extensive, sensitive searches of multiple sources

  5. Methods 2 • Reference set: • Studies included in these reviews and indexed on MEDLINE (n=523) • Filters: • 22 published diagnostic filters • Developed and tested using variety of methods: pragmatic as well as research-based • Obtained from original sources • Checked and ‘translated’ in a standard way to run in Ovid interface

  6. Analysis: stage 1 • Comparison of sensitive searches with restricted searches • For each of the 23 search strategies (sensitive + 22 filters) calculated: • Overall sensitivity = number of relevant studies found/number of studies indexed on MEDLINE • Number needed to read = Number of hits/number of relevant studies found

  7. Result table excerpt: sensitivity

  8. Results: stage 1 • Sensitive searches missed an average of 9% (range 0-13%) of studies indexed on MEDLINE • Searches that included a filter missed between 14 and 58% (range within reviews: 3-88%) of studies indexed on MEDLINE • NNR was reduced from 55 for index test + target condition searches to between 7 and 51 (median 27) for searches that incorporated a filter.

  9. Results: stage 1

  10. Analysis: stage 2 • Sensitive searches used as reference standard, restricted searches evaluated as index test:

  11. Analysis: stage 2 • Calculated: • Comparative sensitivity = a/a+c • Comparative specificity = d/d+b • Plotted data for all reviews combined in receiver operating characteristic (ROC) space to show trade off between sensitivity and specificity.

  12. Results: stage 2

  13. Best performing filters

  14. Conclusions • Sensitive searches • involve large NNRs • miss around 9% of relevant studies indexed on MEDLINE • None of the existing diagnostic filters offers acceptable sensitivity for a substantially reduced NNR • Filters should not be used in to identify studies for inclusion in diagnostic SRs

  15. Plans for further research • Development of new filter with better performance (if this is possible) • Objective method led by data in the records • Investigation of new approach: • Sensitive and multiple database searches compared to • Search fewer databases plus screen references of studies identified • Impact of excluding “harder to find” studies on SR findings

More Related