1 / 57

2006/2007

Estevão, Vânia; Fernandes, Daniela; Leite, Luís; Monteiro, Tiago; Pimenta, André; Rodrigues, João; Silva, Sandra; Teixeira, Hugo;. Class 22. Supervisor: Santos, Cristina. Introdução à Medicina. Quality improvement on the articles that evaluate oncology diagnostic

moeshe
Download Presentation

2006/2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Estevão, Vânia;Fernandes, Daniela;Leite, Luís;Monteiro, Tiago; Pimenta, André;Rodrigues, João; Silva, Sandra; Teixeira, Hugo; Class 22 Supervisor: Santos, Cristina Introdução à Medicina Quality improvement on the articles that evaluate oncology diagnostic tests in high and low impact factor journals 2006/2007

  2. [1] Smidt N, Rutjes A, van der Windt D, Ostelo R, Reitsma J, Bossuyt P, Bouter L, Vet H. Quality of Reporting of Diagnostic Accuracy Studies. Radiology 2005; 235:347-353 Introduction • it is necessary that the results given by a diagnostic test are as much trustful as possible [1] • in oncology, diagnostic tests are essential to determinate if someone has a cancer and how to treat it, if present

  3. [2] Griner PF, Mayewski RJ, Mushlin AI, Greenland P. Selection and interpretation of diagnostic tests and procedures. Principles and applications. Ann Intern Med 1981; 94: 557-592 • when a diagnostic test is used, it is necessary that who uses it knows its risks and benefits and the level of credibility [2] • information obtained from it must be integrated with clinical history and other physical examinations

  4. the combinations between the result given by the diagnostic test and the patient’s real health condition are: Disease present True positive Positive Disease absent False positive Test Disease present False negative Negative Disease absent True negative

  5. [3] Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HCW, for the STARD group. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Family Practice 2004; 21: 4–10 STARD c h e c k l i s t • published by a group of scientists and editors in 2003 • consists of 25 items and encourages the improvement of quality of reporting of studies of diagnostic accuracy [3]

  6. Aim • evaluate how the quality of reporting in oncology diagnostic test accuracy from 2001 to 2005 is developing • compare the evolution in quality of the journals of higher and lower impact factors • evaluate if the journals with lower and higher impact factor publish diagnostic test accuracy articles with different quality

  7. Materials and Methods Study design :Sistematic review Bibliographic search : • we performed a systematic search in all issues of Reporting of Studies of Diagnostic Accuracy in Oncology Literature

  8. we created a query using specific terms (MeSH terms) and words that might be present in articles with interest to our work* • we used ISI Web of Knowledge to obtain the names of the journals which had the impact factor we wanted in each situation * based on a search strategy in PubMed (MEDLINE) for publications about the evaluation of diagnostic accuracy, suggested by a research article: Devillé, W. L. et. al., Conducting systematic reviews of diagnostic studies: didactic guidelines

  9. we repeated 10 times the query, two for each year, and from these one to each group of impact factors • the queries used to each year are the follow:

  10. 2001 – impact factor 1 or less (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND(INT J HYPERTHER[Journal] OR AM J CLIN ONCOL-CANC[Journal] OR SURG ONCOL[Journal] OR TERATOGEN CANCIN MUT[Journal] OR CLIN ONCOL-UK[Journal] OR J EXP CLIN CANC RES[Journal] OR INT J GYNECOL CANCER[Journal] OR NEOPLASMSA[Journal] OR JPN J CLIN ONCOL[Journal] OR ONKOLOGIE[Journal] OR EUR J GYNAECOL ONCOL[Journal] OR BREAST[Journal] OR CURR PROB CANCER[Journal] OR B CANCER[Journal] OR BIODRUGS[Journal] OR PROSTATE CANCER P D[Journal] OR TUMORI[Journal] OR EUR J CANCER CARE[Journal] OR PEDIATR HEMAT ONCOL[Journal] OR ENVIRON CARCIN ECO R[Journal] OR ONCOLOGY-NY[Journal] OR EXP ONCOL[Journal] OR J TUMOR MARKER ONCOL[Journal])

  11. 2001 – impact factor 4 or higher (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND (CA-CANCER J CLIN[Journal] OR J NATL CANCER I[Journal] OR ADV CANCER RES[Journal] OR BBA-REV CANCER[Journal] OR J CLIN ONCOL[Journal] OR CANCER RES[Journal] OR ONCOGENE[Journal] OR EXP CELL RES[Journal] OR CLIN CANCER RES[Journal] OR SEMIN CANCER BIOL[Journal] OR CARCINOGENESIS[Journal] OR GENE CHROMOSOME CANC[Journal] OR LEUKEMIA[Journal] OR INT J CANCER[Journal])

  12. 2002 – impact factor 1 or less (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND(CHEMOTHERAPY[Journal] OR CURR PROB CANCER[Journal] OR ONKOLOGIE[Journal] OR CANCER PRACT[Journal] OR J CHEMOTHERAPY[Journal] OR UROL ONCOL[Journal] OR CLIN ONCOL-UK[Journal] OR J CANCER EDUC[Journal] OR J EXP CLIN CANC RES[Journal] OR JPN J CLIN ONCOL[Journal] OR NEOPLASMA[Journal] OR PEDIATR HEMAT ONCOL[Journal] OR BREAST[Journal] OR PROSTATE CANCER P D[Journal] OR B CANCER[Journal] OR EUR J CANCER CARE[Journal] OR EUR J GYNAECOL ONCOL[Journal] OR TUMORI[Journal] OR EXP ONCOL[Journal] OR J ENVIRON SCI HEAL C[Journal])

  13. 2002 – impact factor 4 or higher (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND (CA-CANCER J CLIN[Journal] OR J NATL CANCER I[Journal] OR NAT REVCANCER[Journal] OR J CLIN ONCOL[Journal] OR ADV CANCERRES[Journal] OR BBA-REV CANCER[Journal] OR CANCER RES[Journal] OR ENDOCR-RELAT CANCER[Journal] OR CLIN CANCER RES[Journal] OR ONCOGENE[Journal] OR NEOPLASIA[Journal] OR SEMIN CANCERBIOL[Journal] OR CARCINOGENESIS[Journal] OR CANCER EPIDEMBIOMAR[Journal] OR EXP CELL RES[Journal] OR LEUKEMIA[Journal] OR CURR OPIN ONCOL[Journal] OR GENE CHROMOSOME CANC[Journal] OR INT J CANCER[Journal] OR STEM CELLS[Journal])

  14. 2003 – impact factor 1 or less (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND(CLIN ONCOL-UK[Journal] OR CANCER PRACT[Journal] OR JPN J CLIN ONCOL[Journal] OR TERATOGEN CARCIN MUT[Journal] OR NEOPLASMA[Journal] OR PROSTATE CANCER P D[Journal] OR PEDIATR HEMAT ONCOL[Journal] OR B CANCER[Journal] OR EUR J CANCER CARE[Journal] OR J EXP CLIN CANC RES[Journal] OR J ENVIRON SCI HEAL C[Journal] OR EUR J GYNAECOL ONCOL[Journal] OR BREAST[Journal] OR J CANCER EDUC[Journal] OR TUMORI[Journal] OR EXP ONCOL[Journal])

  15. 2003 – impact factor 4 or higher (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields]))AND (NAT REV CANCER[Journal] OR CA-CANCER J CLIN[Journal] OR CANCER CELL[Journal] OR J NATL CANCER I[Journal] OR J CLIN ONCOL[Journal] OR ENDOCR-RELAT CANCER[Journal] OR CANCER RES[Journal] OR BBA REV CANCER[Journal] OR ADV CANCER RES[Journal] OR LANCET ONCOL[Journal] OR CLIN CANCER RES[Journal] OR ONCOGENE[Journal] OR STEM CELLS[Journal] OR SEMIN CANCER BIOL[Journal] OR LEUKEMIA[Journal] OR SEMIN ONCOL[Journal] OR CANCER EPIDEM BIOMAR[Journal] OR CARCINOGENESIS[Journal] OR CURR OPIN ONCOL[Journal] OR INT J CANCER[Journal] OR NEOPLASIA[Journal] OR INT J RADIAT ONCOL[Journal] OR GENE CHROMOSOME CANC[Journal] OR CANCER[Journal])

  16. 2004 – impact factor 1 or less (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND(BIRTH DEFECTS RES B[Journal] OR JPN J CLIN ONCOL[Journal] OR INT J BIOL MARKER[Journal] OR CLIN ONCOL-UK[Journal] OR NEOPLASMA[Journal] OR BREAST[Journal] OR B CANCER[Journal] OR TERATOGEN CARCIN MUT[Journal] OR EUR J CANCER CARE[Journal] OR TUMORI[Journal] OR J EXP CLIN CANC RES[Journal] OR EUR J GYNAECOL ONCOL[Journal] OR J CANCER EDUC[Journal] OR PEDIATR HEMAT ONCOL[Journal] OR EXP ONCOL[Journal])

  17. 2004 – impact factor 4 or higher (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields]))AND (CA-CANCER J CLIN[Journal] OR NAT REV CANCER[Journal] OR CANCER CELL[Journal] OR BBA-REV CANCER[Journal] OR J NATL CANCER I[Journal] OR J CLIN ONCOL[Journal] OR EJC SUPPL[Journal] OR LANCET ONCOL[Journal] OR CANCER RES[Journal] OR SEMIN CANCER BIOL[Journal] OR ONCOGENE[Journal] OR ADV CANCER RES[Journal] OR LEUKEMIA[Journal] OR CLIN CANCER RES[Journal] OR STEM CELLS[Journal] OR CARCINOGENESIS[Journal] OR MOL CANCER THER[Journal] OR MOL CANCER RES[Journal] OR ONCOLOGIST[Journal] OR EBDOCR RELAT CANCER[Journal] OR CANCER EPIDEM BIOMAR[Journal] OR CANCER[Journal] OR INT J CANCER[Journal] OR NEOPLASIA[Journal] OR ANN ONCOL[Journal] OR INT J RADIAT ONCOL[Journal] OR GENE CHROMOSOME CANC[Journal] OR ANN SURG ONCOL[Journal] OR EXP CELL RES[Journal])

  18. 2005 – impact factor 1 or less (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND(EJSO[Journal] OR MED ONCOL[Journal] OR EUR J CANCER CARE[Journal] OR CANCER NURS[Journal] OR B CANCER[Journal] OR EXP ONCOL[Journal] OR TUMORI[Journal] OR NEOPLASMA[Journal] OR J CANCER EDUC[Journal] OR J EXP CLIN CANC RES[Journal] OR CURR PROB CANCER[Journal] OR EUR J GYNAECOL ONCOL[Journal] OR PEDIATR HEMAT ONCOL[Journal] OR TERATOGEN CARCIN MUT[Journal])

  19. 2005 – impact factor 4 or higher (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields]))AND (CA-CANCER J CLIN[Journal] OR NAT REV CANCER[Journal] OR CANCER CELL[Journal] OR J NATL CANCER I[Journal] OR BBA-REV CANCER[Journal] OR J CLIN ONCOL[Journal] OR LANCET ONCOL[Journal] OR CANCER METAST REV[Journal] OR SEMIN CANCER BIOL[Journal] OR ADV CANCER RES[Journal] OR CANCER RES[Journal] OR ONCOGENE[Journal] OR LEUKEMIA[Journal] OR STEM CELLS[Journal] OR CLIN CANCER RES[Journal] OR MOL CANCER RES[Journal] OR MOL CANCER THER[Journal] OR ONCOLOGIST[Journal] OR CARCINOGENESIS[Journal] OR ENDOCR-RELAT CANCER[Journal] OR CANCER[Journal] OR INT J CANCER[Journal] OR BREAST CANCER RES TR[Journal] OR INT J RADIAT ONCOL[Journal] OR CANCER TREAT REV[Journal] OR CANCER EPIDEM BIOMAR[Journal] OR ANN ONCOL[Journal] OR PROG EXP TUMOR RES[Journal] OR CELL ONCOL[Journal] OR NEURO-ONCOLOGY[Journal] OR EXP CELL RES[Journal] OR BRIT J CANCER[Journal] OR CANCER IMMUNOL IMMUN[Journal] OR BREAST CANCER RES[Journal])

  20. With these queries , 695 articles were obtained with impact factor 1 or less and 3984 articles were obtained with impact factor 4 or higher

  21. Inclusion criteria • the articles should: • deal with studies that evaluate the accuracy of oncology diagnostic tests - be written in english, french, spanish or portuguese

  22. have an abstract • deal with human research • they must have been published in a journal with impact factor 1 or less or at least 4 and published in the years studied

  23. Exclusion criteria • full article is not available • review articles and case reports will be excluded

  24. Selecting articles • using SPSS we required 10 random samples of 50 articles, 2 for each of the five years (one random sample for each of the impact factors) • the articles were then analysed by title and abstract (when necessary and available) and included or not accordingly the inclusion criteria

  25. when analysing the articles by abstract, as soon as we got 10 articles for the yearand impact factor wanted we stopped our analysis • when the number of articles selected was not the desired we performed new one-on-one random samples until we obtained them

  26. Article’s quality • the article’s quality was determined by the number of points from the STARD checklist present* • each article was submitted to evaluation by two reviewers that independently graded them as far as quality was concerned • * (Bossuyt, P. M. et. al., Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative, Annals of Internal Medicine Vol 138 – nº1, 7 January 2003)

  27. disagreements were solved by a third reviewer • the results obtained were stored in a database on SPSS for posterior analysis

  28. Data extraction • For each article, the information extracted was: - the year of publication - the journal of publication (and its impact factor) - The number of STARD points present

  29. - Statistical methods • the total number of STARD items follows a normal distribution, so we used mean and standard deviation • however, the ANOVA method for homogeneity of variances is not accomplished • this way we used a non-parametric test (Kruskal Wallis)in our analysis to check if the results have statistic significance

  30. Results - Article selection - titles • from the random samples required to SPSS, we obtained 522 articles • after analysing the titles, we selected 202 articles to read the abstracts

  31. the first and second reviewers agreed 492 times – 94% - by accepting 199 articles and rejecting 293. • the level of agreement is shown by the value Kappa = 0,881 • the difference between the 199 included by agreement and the 202abstractsanalysed are the articles included by the third reviewer

  32. - Article selection - abstracts • the 202 articles included by the title were then analysed by abstract • after analysing the abstracts, we selected 103 articles to include in our work: 88 were not accepted and 11 were unavailable

  33. the first and second reviewers agreed 189 times – 94% - by accepting 102 articles and rejecting 87 • the level of agreement is shown by the value Kappa = 0,979 • the difference between the 102 included by agreement and the 103full textsselected is the article included by the third reviewer

  34. - Article analysis – STARD checklist • we had no access to 3 full papers, so we needed 103 articles to collect the 100 we included in the work • we splited the analysis by the two groups of impact factors and the five years analysed • we then analysed both variables togheter to reach some conclusions

  35. - First and second reviewer agreement 1 – Identify the article as a study of diagnostic accuracy (recommended MeSH heading “sensitivity and specificity”) 25 – Discuss the clinical applicability of the study findings

  36. 6 – Data collection: was data collection planned before the index test and reference standard were performed (prospective study) or after (retrospective study) 14 – When study was done, including beginning and ending dates of recruitment

  37. - Global analysis 25 – Discuss the clinical applicability of the study findings 21 – Estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals) 20 – Any adverse events from performing the index tests or the reference standard 24 – Estimates of test reproducibility, typically imprecision (as CV) at 2 or 3 concentrations

  38. - Impact factor 1 or less 25 – Discuss the clinical applicability of the study findings 8 – Technical specifications of material and methods involved including how and when measurements were taken, and/or cite references for index tests and reference standard 17 – Time interval from the index tests to the reference standard, and any treatment administered between 24 – Estimates of test reproducibility, typically imprecision (as CV) at 2 or 3 concentrations

  39. - Impact factor 4 or higher 25 – Discuss the clinical applicability of the study findings 21 – Estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals) 24 – Estimates of test reproducibility, typically imprecision (as CV) at 2 or 3 concentrations 20 – Any adverse events from performing the index tests or the reference standard

  40. - Confronting years and impact factors Mean (standard deviation) of the STARD score of the journals with impact factor ≤ 1 e ≥4 splited by year

  41. - T-Test Group Statistics * these results have statistic significance because p<0,01

  42. Discussion • it is possible to see that in both impact factors the article quality is improving from 2001 to 2005 • the improvement of quality might be as a consequence of the increasing demanding of magazines reviewers that only publish articles with quality

  43. we believe that these improvement is just the beginning of a great improvement in all articles quality in this area • the evolution was higher in the articles with impact factor 1 or less – will they, within some years, have the same quality that the ones with impact factor 4 or higher?

  44. in order to get some definitive conclusion about this, this study should be repeated, for example, within 2 years

  45. it is not possible to conclude if the STARD checklist had influence or not, because: - we only analysed two years after its publication - the differences might have happened as a consequence of a increasing attention with the readers’ demands on quality

More Related