Bibliometric assessment of research performance in social sciences and humanities

1. Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden University, the Netherlands

2. Assessment of research performance: Basic assumptions - 1 • The concept of research quality does have a meaning in all fields of science and scholarship • An assessment methodology should take into account the nature of the field

3. Assessment of research performance: Basic assumptions - 2 • Important contributions to scholarly progress are sooner or later communicated in scholarly publications

4. Research quality: Dimensions • Contribution to scholarly progress • Enlightenment of the general public

5. Science versus humanities (Price) - 1

6. Science versus humanities (Price) - 2 • Different substantive contents ask for different types of social organisation and information exchange • Science: Citation Index • Humanities: Normal archival library

7. Social sciences • Social sciences constitute a heterogeneous domain, with both ‘science-like’ and ‘humanities-like’ orientations • Even sub-disciplines may be heterogeneous (e.g., sociology)

8. Overall ISI coverage by main field

9. Sub-disciplines (non-exhaustive list)

10. ‘Other’ social sciences and humanities • ‘National publication model’ is dominant • Books play an important role • Basic assumptions of a journal citation index are less valid • ISI Indexes have inadequate coverage • They also contain national journals

11. Indicator of a journal’s national orientation (INO) • The share of the papers from the country most frequently publishing in a journal • A purely national journal would have an INO value of 100 per cent

12. National orientation of journals (INO) for 4 fields

13. Policy assumption for social sciences and humanities The extent to which research findings • reach beyond a purely national or local viewpoint • and are exposed to criticisms from a wide international scholarly audience is a relevant criterion of research quality

14. The problem: ISI ≠ ‘International’ • Works exposed to an international audience are not necessarily included in the ISI Indexes • Works included in the ISI Indexes are not necessarily exposed to an international audience

15. This book argues: • It cannot be taken for granted that the ISI Citation Indexes provide such indicators in all subfields of these domains of scholarship • A challenge would be to systematically explore alternative data sources and methodologies

16. 4 Types of bibliometric studies

17. Alternative approaches • Expand the WoS with additional sources • Classification of publications and sources based on scholars’ quality perceptions • Library collection analysis

18. 1. Expand the WoS with additional sources • University of Granada (Spain): • Creation of a citation index with 200 Spanish social science source journals not covered by the WoS • 50,000 uses per year

19. 2. Classifications based on scholars’ quality perceptions • Case study on Flemish Law • Questionnaires; No citation analysis • Publications in Dutch: 81 % • Publications in journals: 59 %

20. Classification of (national) journals based on a questionnaire into: • Scholarly vs. non-scholarly • Outstanding (A), good (B) and less good (C)

21. The ‘best’ indicator of scholarly research performance (in Flemish Law): Count the number of • Single and multi-authored books (first editions only) • PhD theses • Publications with a length > 5 pages

22. 3. Library collection analysis • Focuses on books • Determine the number of academic library copies per book title • Example: Use Worldcat (Linmans, CWTS, 2007)

23. END