EXPERIMENTAL PAPERS IN THE BJS – LESSONS LEARNED AND ROOM FOR IMPROVEMENT

1. EXPERIMENTAL PAPERS IN THE BJS – LESSONS LEARNED AND ROOM FOR IMPROVEMENT Editors Assistant Project Malin Sund 2011

2. Background • The importance, relevance and quality of experimental papers in the BJS are under debate • Lack of clinical relevance – lack of interest among the BJS readership? • Do experimental papers reach the correct readership? • Are the experimental papers a “burden” for the journal and its IF? • What experimental papers are successful in a clinical journal?

3. Definition of an experimental paper All papers in which a novel (not in clinical practice) topic is explored using in vitro, animal and/or human samples

4. Materials & Methods • Experimental papers in the BJS 2004-2008 were reviewed in detail, and a database constructed (145 original articles + 21 leaders/reviews) • Basic article descriptors • Field of surgery, disease/topic, aim, set-up, ethics, statistics • Evaluation of research question – clinical vs. basic science • Citation metrics • Total number and toward IF • Total number and toward IF in surgical journals • Total number and toward IF in non-surgical journals • Similar database for ASO and Annals 2007-2008 (188 original articles) • Citation metrics for J Surg Res 2007-2008 • Statistical analysis using non-parametric ANOVA (Kruskal-Wallis) for multiple groups and Mann-Whitney when comparing two groups

5. BJS 2004-2008 • Number of experimental papers stable • Number of leaders/reviews fluctuate more • Average (SD) number of • figures 3 (1,95) • tables 2 (1,64) • references 30 (8,70) • authors 6 (1,45) • disciplines 2 (1,07) Recommendations to authors 2011: A maximum of 5 figures/tables and 30 references!

6. Where do experimental papers come from? 2004 2008 2005 2006 2007

7. BJS 2004-2008 *** ns • As expected the total average citations 2004 are significantly higher than 2007-2008 • No significant differences in citations within 2 years 2004-2008

8. BJS 2004-2008 • No significant differences among citations among papers in relation to field of surgery • (ANOVA) • In pairwise comparison both GI groups were significantly more cited than Breast/Endo • (p = 0.02) • Most citations total and within 2 years in non-surgical journals • Surgical journals cite experimental papers less • Most surgical citations – general, transplantation, CIT < 2 groups

9. BJS vs. the competition • The number of experimental papers is not significantly different between BJS, ASO and • Annals • J Surg Res has a focus on experimental surgical research and the majority of papers • are experimental

10. BJS vs. the competition *** *** 0,052 * • Both total citations and citations within 2 years (toward IF) are significantly higher for • experimental papers in Annals, when compared to BJS, ASO and J Surg Res • ASO has higher citations on experimental papers than BJS

11. BJS vs the competition • Half of all citations within 2 years. Note J Surg Res! • Most citations total and within 2 years in non-surgical journals • Surgical journals cite experimental papers less

12. BJS CIT ≤ 2 • Only 9 (6.2%) “0-citation” papers among experimental papers in the BJS 2004-2008 • 24 (16%) papers with 2 or less citations (CIT ≤ 2) • 68% of these citations were within 2 years (count toward IF) • Top 10 - Average total citations 41,3 (range 28-80), 27% within 2 years • 53 papers have ≥ 10 citations (37%), 101 ≥ 5 citations (70%) • Most citations in non-surgical journals (74%) • Less citations in surgical journals (26%) BEST IN SHOW

13. BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2 Successful experimental papers TEND to be…. Field of surgery “GI related translational papers on cancer, with a topic related to biomarkers, prognosis prediction or pathogenesis”

14. BJS Top 10

15. BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2 Top 10

16. Experimental paper IF Do the experimental papers have a negative impact in the IF? What would the IF of BJS be based on experimental papers only? An IF factor constructed only on experimental papers is not significantly different than the actual IF of BJS

17. Questions • IF of experimental papers similar to that of the BJS – but would it be better without these? • Likely not… • How to predict what will be highly cited? • GI, surgical device/intervention, cancer, biomarker, translational • Why is there such a difference in papers related to field of surgery? • Low numbers and/or quality on trauma, breast/endo • Does this reflect the overall low number/quality of papers within the field published in the BJS? • How to cut the tail of poor papers?

18. Conclusions • The overall quality of experimental papers is decent • 70% have ≥ 5 citations total, and 27% ≥ 5 citations within 2 years • The papers attract the majority of citations from outside the surgical readership – good/bad? • Citations totally and within two years are low in surgical journals • Translational papers attract most citations – perceived higher clinical relevance?

19. Possibilities • Focus on translational papers with a clear clinical relevance – clinical relevance box • Attract more experimental research on trauma, breast/endo • GI papers highly cited - maintain position • Cancer and surgical interventions/device related papers are highly cited • The experiments and set-up needs to be scrutinized - Editors with specific interest in experimental research?

20. Acknowledgements • Supervisor Kjetil Søreide – Tack, tack! • Bibliometrics Gavin Stewart /Wiley • Editorial Team for good discussions