1 / 29

Does It Matter Where We Publish?

Does It Matter Where We Publish?. Adam Eyre-Walker University of Sussex. Reasons. Prestige peers employers Impact scientific community society. Measuring impact. General difficult to measure Citations Easy to measure Velvet Underground effect

sally
Download Presentation

Does It Matter Where We Publish?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Does It Matter Where We Publish? Adam Eyre-Walker University of Sussex

  2. Reasons • Prestige • peers • employers • Impact • scientific community • society

  3. Measuring impact • General • difficult to measure • Citations • Easy to measure • Velvet Underground effect • “only sold 10,000 copies, but everyone who bought it formed a band” (Brian Eno)

  4. Question Do papers of similar quality garner more citations in better quality journals?

  5. Ideal experiment • Papers made anonymous • authors • journal • structure • Assessed for quality by experts • Assessments confidential • Citations tracked

  6. Research Assessment Exercise (RAE) • Each department submits • 4 publications per academic (75%) • 4 indicators of Esteem (5%) • Environment • PhD student numbers • Grant income • Research structure • Facilities • Superceded by REF

  7. RAE dataset • 1170 papers published 2001 - 2007 • Scored by AEW • Scored 1 to 4 (or unclassified) • Subjects : evolutionary biology, genomics, bioinformatics, ecology, animal behaviour and organismal biology • Ranked without reference to • journal • citations

  8. RAE scores

  9. Journal quality • Impact factor • Number times papers published in the two previous years are cited in target year • Two components • Quality of papers • Effect of the journal on citations

  10. IF categories

  11. Citations • No. of citations from ISI Web of Science • October 2008

  12. Data • For each paper • Year of publication • RAE score (RS) • No. of citations • Impact factor of journal (IF category) • Otherwise anonymous

  13. How good is the assessor? • Correlation between AEW and co-scorers > 0.7 • Correlation between RS and no. of citations within each IF category and within each year

  14. How good is the assessor? • Correlation between AEW and co-scorers > 0.7 • Correlation between RS and no. of citations within each IF category and within each year • Average r = 0.27 (p<0.001) • Average rmax = 0.73

  15. RS v citation correlation r = -0.37 p = 0.005

  16. Central question • Do papers of similar quality accumulate more citations in some journals than others?

  17. RAE score 2 in 2003

  18. RAE score 2 p < 0.001

  19. RAE score 3 p < 0.001

  20. ANCOVA

  21. ANCOVA

  22. Alternative explanation • Systematically underestimate quality of papers in high ranking journals • Test • Assume no effect of journal • Consider correlation between RS and journal (IF) controlling for citations

  23. Partial correlations

  24. ANCOVA

  25. Question • Impact factors – relative contribution of • Paper quality • Journal effect

  26. Summary • Assessor scores correlated to citations • strength correlated to age • Assessor scores influenced by journal • Papers published in higher quality journals accumulate more citations independent of quality • F1000 data

  27. Question • Where will we submit?

  28. Thanks Nina Stoletzki

  29. Assessor and journal • Assume • citations measure quality • no journal effect

More Related