1 / 33

Jørgen Hilden

GRAPHICAL AND CLUSTER-ANALYTIC TECHNIQUES FOR PRELIMINARY INSPECTION OF DIAGNOSTIC TEST EVALUATION STUDIES PRIOR TO A META-ANALYSIS. Jørgen Hilden

cruz
Download Presentation

Jørgen Hilden

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRAPHICAL AND CLUSTER-ANALYTIC TECHNIQUESFOR PRELIMINARY INSPECTION OFDIAGNOSTIC TEST EVALUATION STUDIESPRIOR TO A META-ANALYSIS Jørgen Hilden Univ. of Copenhagen Dept. of Biostatistics Copenhagen, Denmark ISCB Szeged 2005

  2. GRAPHICAL AND CLUSTER-ANALYTIC TECHNIQUESFOR PRELIMINARY INSPECTION OFDIAGNOSTIC TEST EVALUATION STUDIESPRIOR TO A META-ANALYSIS IN THIS VERSION OF THE SLIDES I HAVE ADDED EXPLANATORY NOTES ALONG THE WAY IN THE PowerPoint NOTES FIELDS Jørgen Hilden Univ. of Copenhagen Dept. of Biostatistics Copenhagen, Denmark ISCB Szeged 2005

  3. Until recently, the Cochrane Collaborationwas 100% interventional research.Now, a Cochrane Handbook for systematic reviews and meta-analyses of diagnostic studies is under way. For the time being, endeavours are restricted to the classical Black-and-White approach, i.e., 2-by-2 tables summarized in terms of sensitivity, specificity, etc.

  4. Fairly sophisticated procedures are here available for meta-analysis[Chapter 8 of the Handbook / C. Gatsonis et al.]But something is missing !TECHNIQUES FOR PRELIMINARY INSPECTION of the “raw” data, i.e., source study 2-by-2 tables + descriptors oftechnical & clinical aspects of each study

  5. You will agree with me thatgraphical and descriptive (“data-analytic”) procedures for preliminary mustering of the data form an indispensable part of statistical craftmanship. Checking for oddities and outliers…

  6. Diagnostic studies are highly variable in scope and sophistication*  oddities are frequent and important to detect *to put it politely

  7. One may want to see an array of 2-by-2 tables, arranged by summary statistics, or arranged according to descriptors of the technical & clinical setting • One may want to visualize a clustering of primary studies that throws light on heterogeneity – and its causes

  8. Displaying 2-by-2 tables Despite their key rôle in epidemiology, even epidemiologists do not have any standards how to display and visually compare such tables. ►Two main challenges that I see: – The “observations” are inherently 3- or 4-dimensional [ROC diagrams show only 2 d’s] – Near-zero frequencies are hard to distinguish but differences may be crucial

  9. Read off: Fraction diseased Sensitivity Specificity

  10. False positives upper rightfalse negativeslower left(as before)Paddocks | for the black sheep|

  11. False pos/neg minorities still in area-truerepresentation; but linear pen sizeis ~ sqroot(#).False positives upper rightfalse negativeslower left(as before)The black sheep .dropped something .

  12. Note 1.96 SE marks (all approx. equal)

  13. Variant: draw confidence ellipses <clutter!> Variant: ”low-quality” source studies in GREY

  14. ” Comet graph” ROC and ”posterior” counterpart (PV’s)

  15. CLUSTER-ANALYTIC TECHNIQUES Exploring heterogeneity – there are endless variants

  16. faneg fapos true neg true pos

  17. Five studies used a different protein marker * * * * *

  18. To reduce clutter, should only comparatively homogeneous groups be shown? The black-sheep plots are perhaps not so useful for hierarchical graphs

  19. Variant using 2X2 table distance horiz. & vertically

  20. Those inter-study discrepancies: you may wonder which ones are at all statistically significant? – The majority! The next graph pushes the study tree to the bottom and displays the 16 lowest inter-study chi-squares. Out of 78 pairs, only 9 (16) are not significant at the 0.05 (0.001) level.

  21. Graphical presentation ofmeta-analytical results A well-known device is to show summary estimates along with the source studies’ estimates [examples already shown] But beware of the fixed-effect fallacy / heresy. How do we summary-display the inter-centre variation? Think!

  22. Outlook • Audiences are conservative when it comes to inspecting data in novel ways, and • the graphs that one person finds informative others find unintelligible. Also, • clinical problems, with their human and economic stakes, are so diversified. So, a spectrum of graphical tools ought to be made available to diagnostic test evaluators.

  23. IS THE SUN RISING? I HOPE SO j.hilden@biostat.ku.dk

  24. IS THE SUN RISING? Thank you for your attention. Comments are most welcome. I HOPE SO j.hilden@biostat.ku.dk

More Related