1 / 31

Evaluative Bibliometrics: Citation based Research Indicators

Evaluative Bibliometrics: Citation based Research Indicators. Bo Jarneving Bibliometrifunktionen GUB 2011. Variables for citation based performance indicators. Document type articles, reviews, letters, notes Observation period Citation interval Self-citations With or without

nitesh
Download Presentation

Evaluative Bibliometrics: Citation based Research Indicators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluative Bibliometrics: Citation based Research Indicators Bo Jarneving Bibliometrifunktionen GUB 2011

  2. Variables for citation based performance indicators • Document type articles, reviews, letters, notes • Observation period Citation interval • Self-citations With or without • Subject classification Generally based on ISI ”Journal Subject Categories” Classification of papers published in multidisciplinary journals • Counting method decisions How to count co-authored papers? How to count when there are multiple author affiliations? How to count when there are multiple subject categories?

  3. Observation period • Observation period • 2004-2009 • Variable or fixed citation interval (window)

  4. What is a self citation? Schubert A, Glanzel W, Braun T (1989) Scientometric Datafiles - A Comprehensive Set of Indicators 0n 2649 Journals and 96 Countries In All Major Science Fields and Subfields 1981-1985. Glanzel W, Thijs B, Schubert A, Debackere K(2009) Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance.

  5. Journal Subject Category • A proxy for subject field • Sometimes multiple assignments of journals to categories • Based on citation data and ”manual methods” • A total of 173 categories in the science volume of Journal Citation Report (JCR) • Alphabetical order – no hierarchy

  6. Classification of multidisciplinary papers • Reversing pathological neural activity using targeted plasticity • NATURE, 2011, V470, N7332, P101-U114

  7. Counting method: fractional counts or whole counts • Whole counts • Every author, institution, department and country is assigned one full paper • Fractional counts • A paper is divided up over its authors • If a paper has 3 authors, every author is assigned 1/3 of the paper. • If 2 of these authors are associated with institution A and 1 with institution B,A recieves2/3 of the paper and B 1/3.

  8. Counting methods: fractional counts or whole counts When more than one institutional affilation of an author: • Whole counts • One author associated with two institutions – each insitution recieves one full paper • Fractional counts • One author associated with two institutions – each insitution recieves one half of a paper

  9. Counting methods: fractional counting of papersExample 1: 1 paper, 1 category, 2 authors, 3 institutions (A-C) 1c 1/4 1/2 Agets Bg Cg 1/4 1/2 1/2

  10. Counting methods: fractional counting of papersExemple 2: 1 paper, 2 categories (1, 2), 2 authors, 3 institutions (A-C) 1/4 1/4 1/8 1/2 1/4 1/8 Category2 Agets Bgets Cgets Category 1 Agets Bgets Cgets 1c 1/4 1/4 1/2 1/4 1/8 1/8

  11. Counting methods: fractional counting of citations.Example 3: 1 paper, 1 category, 2 authors, 3 institutions (A-C).The paper has recieved 10 citations (c). ¼ x 10 1c 1/2 Arecieves 2.5 c Brecieves 2.5 c Crecieves 5 c ¼ x 10 1/2 ½ x 10

  12. Counting methods: fractional counting of citationsExample 4: 1 paper, 2 categories (1, 2), 2 authors, 3 institutions (A-C). The paper has recieved 10 citations (c) ¼ x 10 ¼ 1/8 x 10 ½ ¼ Category 2 Arecieves Brecieves Crecieves Category 1 Arecieves Brecieves Crecieves 1/8 x 10 1c ¼ x 10 ¼ ½ ¼ 1/8 x 10 1/8 x 10

  13. Counting methods: whole counts of citationsExample 5: 1 paper, 2 categories (1, 2), 2 authors, 3 institutions (A-C). The paper has recieved 10 citations 1 x 10 1 x 10 1 x 10 Kategori 2 Ar Br Cr Kategori 1 Ar Br Cr 1 x 10 1c 1 x 10 1 x 10 1 x 10 1 x 10 1 x 10

  14. Normalization of citation counts • Observed value: institutional (departmental) citation counts • Expected value: approximated world averages • based on Web of Science citation data

  15. Normalization of citation counts The expected value (reference value) • Define groups on basis of the institutional publication pattern. Group by: • Publication date (year) • Journal Subject Category • Document type • For each such group, compute the Field Citation Score (FCS) • Based on citations between papers indexed in WoS • Approximations of world-citation-averages for each group

  16. Normalization of citation counts: defining groups • Example 1: The department (group) publishes in journals assigned to one journal subject category only (neurosciences). • Brain Research 4 articles 2004 • Brain Research 1 letter 2004 • Journal of Neuroscience 3 articles 2002 • Journal of Neuroscience 1 review 2003 • All authors are from one department • Gather all articles published in journals assigned to neurosciences and published 2004 • Gather all letters published in journals assigned to neurosciences and published 2004 • Gather all articles published in journals assigned to neurosciences and published 2002 • Gather all reviews published in journals assigned to neurosciences and published 2003

  17. Normalization of citation counts: reference values For each group (1- 4) compute the world-average number of citations (Field Citation Score [FCS]): • Neurosciences-articles 2004 FCS = 4.2 • Neurosciences-letters 2004 FCS = 1.0 • Neurosciences-articles 2002 FCS = 3.8 • Neurosciences-reviews 2003 FCS = 8.5

  18. Normalization of citation counts: the department’s counts

  19. Normalization of citation counts: comparing Pic 20

  20. Normalization of citation counts: averages • CPP = Citation Per Publication (department) • FCSm = Mean Field Citation Score • CPP/FCSm = the Crown Indicator • Compute CPP: (3 x 7 + 1 x 6 + 2 x 5 + 1 x 4 + 2 x 1) / (3+1+2+1+2) = 4.78 • Compute FCSm: (4 x 4.2 + 1 x 1.0 + 1 x 8.8 + 3 x 3.8) / (4 + 1 + 1 + 3) = 4.22 • Compute CPP/FCSm: 4.78 / 4.22 = 1.13

  21. Normalization of citation counts: defining groups • Example 2: The department (group) publishes in journals assigned to more than one journal subject category. • Brain Research 4 articles 2004 • Brain Research 1 letter 2004 • Journal of Neuroscience 3 articles 2002 • Neuroimage 1 article 2003 Four classifications: • Gather all articles published in journals assigned to neurosciencesand published 2003 • Gather all articles published in journals assigned to neuroimagingand published 2003 • Gather all articles published in journals assigned to Radiologyand published 2003 • Gather all articles published in journals assigned to Nuclear Medicine & MedicalImagingand published 2003

  22. Normalization of citation counts: reference values • For each group (1- 7) compute the world-average number of citations (Field Citation Score [FCS]): • Neurosciences-articles 2004 FCS = 4.2 • Neurosciences-letters 2004 FCS = 1.0 • Neurosciences-articles 2002 FCS = 3.8 • Neurosciences-articles 2003 FCS = 5.5 • Neuroimaging-articles 2003 FCS = 3.2 • Radiology-articles 2003 FCS = 2.8 • Nuclear Medicine & MedicalImaging articles 2003 FCS = 4.6

  23. Normalization of citation counts: comparing

  24. Normalization of citation counts: averages • CPP = Citation Per Publication (department) • FCSm = Mean Field Citation Score • CPP/FCSm = the Crown Indicator • Compute CPP: (2 x 7 + 1 x 6 + 2 x 5 + 1 x 4 + 2 x 1 + ¼ x 7 + ¼ x 7 +¼ x 7+ ¼ x 7) / (2+1+2+1+2+¼ +¼ +¼ + ¼) = 4.78 • Compute FCSm: (4 x 4.2 + 1 x 1.0 + 3 x 3.8 + ¼ x 5.5 + ¼ x 3.2 + ¼ x 2.8 + ¼x 4.6) /(4 + 1 + 3 + ¼ + ¼ +¼ + ¼) = 3.69 • Compute CPP/FCSm: 4.78 / 3.69 = 1.30

  25. Normalization of citation counts: paper by paper • Normalization on paper level: 10.48 / 9 = 1.16 • Ratio of averages vs. average of ratios

  26. Total performance indicator • The average impact times the total number of papers: • Where ci is the citation score for paper i, ei the expected number of citations for paper i and N the total number of papers for a department

  27. A real-life example • TI- Engineering of a novel Saccharomyces cerevisiae wine strain with a respiratory phenotype at high external glucose concentrations • JN - Applied and Environmental Microbiology • SC- Biotechnology & Applied Microbiology; Microbiology • DT- article • PY- 2005 • AU- Cecilia Henricsson (Institutionen för kemi- och bioteknik, Chalmers) Maria de Jesus-Ferreira (-) Kristina Hedfalk (Institutionen för kemi) Karin Elbing (-) Christer Larsson (Institutionen för kemi- och bioteknik, Molekylär bioteknik, Chalmers) Joakim Norbeck (Institutionen för kemi- och bioteknik, Molekylär bioteknik, Chalmers) Roslyn Bill (-) Stefan Hohmann (Institutionen för cell- och molekylärbiologi) Lena Gustafsson (Institutionen för kemi- och bioteknik, Molekylär bioteknik, Chalmers)

  28. A real-life example Bibliometric data: • FCS or = field citation score (reference value), self-citations excluded • C = observed citations, self-citations excluded • Observation period 2004-2009 • A citation window of 5 years • Fractionalized author counts • Fractionalized subject category counts • Journal subject categories: • Biotechnology & Applied Microbiology • Microbiology

  29. A real-life example Computing the reference value: • Find all papers published in journals assigned to Biotechnology & Applied Microbiology and/or Microbiology during 2005-2009. • Count all citation to these papers • Fractionalize in case of multiple journal subject category assignments • Compute FCS or : • for Biotechnology & Applied Microbiology , and • for Microbiology. • N = 247 (journals)

  30. A real life example

  31. How to affect the average field-normalized citation score? • Publish in journals covered by Thomsen ISI (SCOPUS) • Publish in high esteem journals • Use a standard departmental name • Use a standard institutional name

More Related