1 / 24

RESEARCH EVALUATION - THE METRICS

RESEARCH EVALUATION - THE METRICS. UNITED KINGDOM OCTOBER 2010. DO WE NEED MORE METRICS? 1962. METRICS ARE HERE TO STAY! 2010. CITATION METRICS. What Types of Data are best for which Purposes? There are no all-purpose indicators

feryal
Download Presentation

RESEARCH EVALUATION - THE METRICS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010

  2. DO WE NEED MORE METRICS? 1962

  3. METRICS ARE HERE TO STAY! 2010

  4. CITATION METRICS • What Types of Data are best for which Purposes? • There are no all-purpose indicators • Start by identifying the question the results are supposed to answer, then collect data • Clearly define • Purpose of the evaluation • Types of data required • How the results will be used

  5. WHAT DO INSTITUTIONS WANT TO FIND OUT FROM CITATION METRICS • What is the university’s research performance? • Are we competitive compared with our peers? • How can the university forecast growth? • Which are our centers of excellence? • What is our citation ranking? • What is the influence of our research? • Which are our most influential papers? • Which are our top researchers?

  6. PRODUCTIVITY • Publication counts – most basic bibliographic measure • Identifies areas of research intensity by field

  7. PRODUCTIVITY – INSTITUTIONS

  8. INFLUENCE – CITATIONS

  9. INFLUENCE – CITATIONS Number of citations to North American scientific papers Source: Thomson Reuters U.S. and Canadian University Science Indicators

  10. INDIRECT INFLUENCESECOND GENERATION CITATION COUNTS • Sum of citation counts for articles citing target paper • Measures long term impact of a paper • Similar to Google PageRank but uses citations, not clicks Thomson Reuters Web of Science

  11. EFFICIENCY H-INDEX • Distribution based indicator • Attempt to reflect productivity and influence • Does not control for time period and field Thomson Reuters InCites

  12. EFFICIENCY % PAPERS CITED – COMPUTER SCIENCE

  13. 2008 2009 2010 2006 2007 All Previous Years Citations Source paper – published in 2009 Cited reference – published in 2007 or 2008 EFFICIENCYJOURNAL IMPACT FACTOR 2009 Impact Factor

  14. Citations in 2009 To items published in 2008 = 156 To items published in 2007 = 210 Sum = 366 Number of items Published in 2008 = 97 Published in 2007 = 85 Sum = 182 CALCULATING 2009 JOURNAL IMPACT FACTOR HEALTH ECONOMICS 366 182 = 2.011

  15. EFFICIENCYJOURNAL IMPACT FACTOR

  16. INTRODUCTION TO THE IMPACT FACTOR NATURAL FLUCTUATIONS OVER TIME

  17. EVALUATING RESEARCHLIMITATIONS OF THE IMPACT FACTOR

  18. EFFICIENCY5-YEAR IMPACT FACTOR Citation rates to journals in the ‘Acoustics’ category are maintained thus increasing the 5-year Impact Factor relative to its 2-Year counterpart

  19. EFFICIENCY5-YEAR IMPACT FACTOR The opposite picture in ‘Genetics and Heredity’ demonstrates a shorter citation lag

  20. USING THE IMPACT FACTOREVALUATING JOURNALS • Appropriate use • To evaluate journals • Misuse • Evaluation of individual articles • Evaluation of institution or researcher

  21. USING THE IMPACT FACTOR MISUSE: EVALUATING INDIVIDUAL PAPERS 30% of articles in Food Policy were not cited at all Journal Impact Factor = 2,011 Source: Thomson Reuters Journal Citation Reports

  22. BENCHMARK YOUR PAPERS AGAINST GLOBAL AVERAGES – IS THIS A HIGHLY CITED PAPER? This article is ranked in the 12,92nd percentile in its field by citations Articles published in ‘Blood’ from 2004 have been cited 34,30 times Hematology articles from this year have been cited 18,83 times This paper has received 40/34,30=1,17 times the expected citations for this journal This paper has received 40/18,83=2,12 times the expected citations for this subject category

  23. WHICH ARE OUR CENTRES OF EXCELLENCE? Dept. Mechanical Engineering Dept. Medicine

  24. PERCENTILE INDICATORS • Paper percentile • Determine percentage of papers at each level of citation

More Related