Research Assessment and Funding: the experience of other countries

1. Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol

2. Change in RAE gradings% of top grades (5 & 5*) from 1992 to 2001 by subject area

3. The Research Game in Academic Life (Lucas, 2006) • Intensification of the management and organisation of research activities • Differentiation of academics within departments over status/workloads • Struggles over classification of research active/research inactive • Lack of value perceived for teaching and associated work • Prioritising of research areas that can attract high levels of research funding and would be worthy of submission to high ranking journals

4. UK University Core Funding for Research and Teaching 2005-6Source: Higher Education Funding Council (HEFCE)

5. Focus of Presentation • What can be learned from other national systems of funding and evaluating research in universities that utilise metrics? • Can other national systems provide alternative ideas to inform the development of the UK system of funding and evaluating research?

6. University State Funding and the Higher Education Landscape in the 21st Century • Reduction in State funding to universities • Resource Dependency Theory (Slaughter & Leslie, 1997) • Global Competition • Creating ‘World Class’ Universities (global league tables) • Globalisation and Higher Education • Convergence of Higher Education Policies?

7. Measuring the Scientific Impact of Nations?

8. National Case Studies of Research Funding and Evaluation • Hong Kong • The Netherlands • Australia and New Zealand

9. Higher Education System in Hong Kong • Eight institutions • Funded by the University Grants Committee (UGC) “As far as the UGC is concerned, the UGC advocates role differentiation among institutions, and the concentration of resources to reward performance and encourage the growth of centres of excellence.” (UGC, Facts and Figures, 2002)

10. Hong Kong RAE 2006 • Based originally on the UK RAE • Single quality threshold at attainable level of excellence, no grading • Carnegie Classifications: • scholarship of discovery • Scholarship of integration • Scholarship of application • Scholarship of teaching

11. Hong Kong System: key issues • Single quality threshold – no differentiation between research ‘active’ and ‘inactive’ staff. • Allowing for multiple dimensions of excellence by broadening the definition of ‘research’ to forms of scholarship. • BUT • Role differentiation amongst institutions mandated. • Greater compulsion for EVERYONE to do research. • Multiple dimensions of evaluation less easy in practice. • Pressure to publish ‘internationally’.

12. Higher Education System in the Netherlands “The universities prepare students for independent scientific work in an academic or professional setting and the hogescholen prepares students to practise a profession and enable them to function self-consciously in society at large.” • 13 universities and 50 hogescholen • Source: Boezerooy, P (2003)Higher Education in the Netherlands: Country Report, CHEPS

13. Research Funding and Evaluation in the Netherlands (VSNU) • Universities are funded for teaching and research as a block grant based primarily on historical circumstances • Development of Research Institute and Schools Structure (separate within Faculties) • Research (mainly self-evaluation) based on: • Quality • Productivity • Relevance • Vitality and Feasibility

14. The Dutch System: key issues • Funding not linked to evaluation of research work • Self-evaluation of research • Research School/Institute structure means that academics can be either research or teaching only • BUT • Binary system/small university sector • Institutions use evaluation to manage, control and direct research activities • Possible separation of teaching and research staff within faculties but also more collaboration of staff across institutions

15. Research Funding and Evaluation in Australia (2008?) • 40 universities • Imminent change to the Research Quality Framework (RQF) – combining quantitative and qualitative evaluation • On going debates on the formation of the RQF • Concern with ‘basket of metric indicators’ – metrics working group • Concern with Impact of research (social, economic, environmental) – impact working group

16. Distribution of publication output by field, Australian universities, 1999-2001 (Butler, 2006)

17. Research Funding and Evaluation in New Zealand (2006, 2012) • Performance-based Research Funding (PBRF) – quantitative and qualitative indicators • Quality of researchers (60%) • Reflect research degree completions (25%) • Reflect external income (15%) • 45 institutions • Some Key Aims of the PBRF • Ensure that research continues to support degree and postgraduate teaching • Prevent undue concentration of funding that would undermine research support for all degrees

18. What can be learned from other national systems that utilise metrics? • Quantitative measures should inform qualitative judgement not replace it. • A ‘basket of metrics’ needs to be utilised rather than simplistic single indicators. • Perverse research practices can be encouraged where simplistic measures are utilised (Butler, 2003).

19. Can other national systems provide alternative ideas? • Widening the definition of research and encouraging the integration of research and teaching • Encouraging greater collaboration across the sector to build capacity • Research for whom? Impact and communication of research • Importance of combined quantitative and qualitative indicators and variety of indicators to be utilised • Indicator utilised that are appropriate for different subject areas • Holistic approach – concern to support the whole higher education sector

20. Dr Martin (Biology, Golden County University) “I would rather see all the vice chancellors lined up for a hundred yards dash and just assign money on that basis, because that exercise would take at the most two minutes. Even the weakest vice chancellor could do a hundred yards in two minutes and then get on with life. It is about as rational as that. At least you could train your vice chancellor and pick a healthy one. At least you would have a use for a vice chancellor at long last, you’d be able to select on a rational basis. It might televise well and you might get money from the rights on watching it. And you might get rid of a few each time.”

21. References • Butler, L. (2003) Explaining Australia’s increased share of ISI publications – the effects of funding formula based on publication counts, Research Policy, 32: 143-155. • Butler, L. (2006) Research Assessment: moving beyond journal outputs, SPRU 40th Anniversary Conference – The Future of Science, Technology and Innovation Policy, University of Sussex, 11-13 September. • DEST, Research Quality Framework, Australia http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research_quality_framework/default.htm • King, D.A. (2004) The Scientific Impact of Nations, Nature, 430 (15th July): 311-6. • Lucas, L. (2006) The Research Game in Academic Life, Maidenhead: SRHE/Open University Press • Slaughter, S. & Leslie, L. (1997) Academic Capitalism: politics, policies and the entrepreneurial university, Baltimore: The John Hopkins University Press. • Tertiary Education Comission, Performance-based Research Fund, New Zealand, http://www.tec.govt.nz/funding/research/pbrf/pbrf.htm • UGC (2002) Facts and Figures, Hong Kong SAR.