1 / 14

Research Excellence Framework

Research Excellence Framework. Position at October 2009 David Otley Distinguished Professor of Accounting & Management Lancaster University Management School Chair Main Panel I for RAE 2008. Issues with RAE 2008. This is seen by HEFCE / government as: Too burdensome

heman
Download Presentation

Research Excellence Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Excellence Framework Position at October 2009 David Otley Distinguished Professor of Accounting & Management Lancaster University Management School Chair Main Panel I for RAE 2008 REF October 2009

  2. Issues with RAE 2008 • This is seen by HEFCE / government as: • Too burdensome • Mainly for them and university administrations • But also for the panels • Too incestuous • Captured by academics • Not enough attention given to impact • Problematic funding outcomes for old 5*s • ‘A few simple metrics’ should replace it REF October 2009

  3. Other issues with RAE 2008 • Past sell-by date • Burden on panels to produce profiles • Game-playing helped by early publication of rules and procedures • Late introduction of sub-profile reporting • Strong advice not to use citation data • B&M panel over-stretched ‘Never again’ as concluding motto REF October 2009

  4. The REF – likely ground rules • Much very similar to RAE • Work associated with individuals • Selective inclusion of staff by universities • 4 (or perhaps 3) research outputs • Profiles will be used • Much the same panel structure • Although B&M (and A&F) need re-organization • Panels will exercise judgment, informed by other data REF October 2009

  5. REF consultation • Consultation document now issued for response by mid-Dec 2009 • Informed by Expert Advisory Groups • Much already non-negotiable • Need to be acceptable to government • Context of likely reduced public expenditure • But some important changes, and issues for subject groups to respond to consultation REF October 2009

  6. Government Policy • A strong and innovative national research base is essential to support national prosperity in a globalised knowledge based economy • Need to strengthen links between investment in research and the economic and social benefits it delivers: • Creating new businesses and improving performance • Developing new products and services • Highly skilled and educated workforce • Improving public services and public policy (Science and innovation investment framework 2004-14) REF October 2009

  7. The REF framework Outputs Impact Environment Quality of the research environment: assessed through narrative and indicators Quality of outputs: assessed through a combination of bibliometrics and expert review Engagement with users: assessed through narrative and indicators Impact of research: assessed through a portfolio of evidence 25% 15% Weights: 60% REF October 2009

  8. Three components • Outputs • Much as before but supplemented with bibliometric information if recommended • Need for careful benchmarking • Need for adequate database • Maybe only 3 items • New 4* definition • Environment • Much as before, but now includes ‘esteem’ and ‘engagement’ REF October 2009

  9. Impact • New area of ‘impact’ • Includes ‘impact’ but ‘engagement with users’ now under ‘environment’ • Has to be based on research work, but current period impact can be based on prior period work • Not necessarily specific papers, more a body of work in a department • Assessed by ‘case examples’ REF October 2009

  10. Other issues • Profiles • Often dichotomised with single point alternative but continuous scale possible (e.g. 2.4) • Same for all three areas? • Need for point definitions • Funding • Gradient of funding line (less in 2008), but contentious especially with reductions likely REF October 2009

  11. Incentive effects [Van der Stede, 2007] Effort “Productive “ effort “Perverse” effort Incentive strength REF October 2009

  12. Conclusions • Need to respond to consultation • Role of bibliometric data • Information requirements • Assessment of impact • Determination of panel composition • Funding principles (economies of concentration ???) REF October 2009

  13. Citation issues • Different discipline features • Average citation rates • Citation lags and half-life • Databases variable quality & coverage • Issues for attention • Inclusion of comments with papers? • Provision of comparative information • Use in esteem measures REF October 2009

  14. Bibliographic issues • Use of journal ‘lists’ • Impact factors • Imputes quality of journal to paper • Maybe ok in aggregate? • Individual panels may well differ in approach • Allocation of people to subject panels? REF October 2009

More Related