The REF assessment framework

1. The REF assessment framework AAH National Conference March-April 2011

2. REF is not RAE

3. Overall assessment framework Main panel criteria and working methods Sub-panel specific input The assessment framework The REF is a single assessment framework, within which panels have flexibility to develop aspects of the criteria and working methods

4. Key documents During 2011, our task is to produce two key documents: • Guidance on submissions • Panel criteria and working methods • These will set in stone what HEIs need to include in their submissions, and how the panels will assess them

5. 2011 • Impact decisions (Feb) • Panel membership (Feb) • Guidance on submissions (Jul) • Draft panel criteria (Jul) • 2012 • Final panel criteria and methods (Jan) • Test submissions system • 2013 • Submissions deadline(Nov ) • Recruit additional assessors • 2014 • Assessment • Publish outcomes (Dec) Timetable

6. Overall quality 15% 20% 65% Outputs Environment Impact ‘rigour, originality and significance’ ‘vitality and sustainability’ ‘reach and significance’ Maximum of 4 outputs per researcher Template and data Template and case studies The overall assessment framework

7. 4* 3* 2* 1* u/c Overall Quality Profile 20% 15% 65% 12 36 42 10 0 Quality Level % of the submission Environment Impact Outputs

8. Submissions Each submission to a UOA will include staff selected by the HEI, their research outputs and associated environment and impact: • Normally one submission by an HEI to a UOA • Multiple submissions in exceptional cases • Joint submissions • Sub-panels to produce UOA descriptors and boundaries

9. Research staff Submissions to include: • Details of eligible staff selected by the HEI • Up to 4 outputs per selected researcher • The number of outputs may be reduced without penalty for staff with circumstances that constrained their productivity: • Early career researchers • Clearly defined periods of absence • Complex circumstances

10. Research outputs • All research outputs placed in the public domain during the publication period (1 Jan 2008 to 31 Dec 2013) will be eligible • Outputs from all types of research to be treated equally • Main panels to expand on: • The diversity of expected output types • How co-authorship will be handled • Requirements for additional information • Criteria for ‘double-weighting’ outputs of exceptional scale and scope

11. Citation data We expect some panels will use citation information to support their review of outputs: • We will identify a standard approach to making data available to panels and provide guidance: • Judgements based on expert review; citations only inform ‘academic influence’ • All types of research to be treated equally • Clear limitations for recently published outputs • Main panels to: • Decide whether or not citation data will be used • Be explicit about any use of citation data

12. Research environment • A structured template to describe, for example: • Overview • Strategy • People • Income, infrastructure and facilities • Collaboration and contribution • Standard data on research income and research students • Main panels to explain: • What kinds of evidence or information are required • The relative importance of each section • The approach to forming an environment ‘sub-profile’

13. Research impact • A broad generic definition of (non academic) impact • Sub-panels will make judgements based on: • Case studies detailing specific impacts • Information about how the unit has supported and enabled impact • Generic templates and scale for the number of cases • Main panels to expand on: • How the criteria (reach and significance) will be applied • Types of impacts and evidence appropriate to their disciplines • Suitable evidence of underpinning research quality • How the ‘impact template’ will influence the sub-profile

14. Equalities and diversity We have built on the measures taken in the 2008 RAE: • A consistent approach to enabling staff with constraining circumstances to submit fewer outputs • All institutions to adhere to a code of practice on the fair and transparent selection of staff • Evidence of support for equalities and diversity (within the research environment) • Analysis of selection rates at sector level • Panels will be briefed on equalities • Ongoing advice from an equalities expert panel

15. Panel working methods Working methods to explain: • How the main panels will in practice carry out their roles in providing oversight and governance • How the sub-panels will in practice carry out the assessment, including: • The approach to making use of assessors • The approach to allocating and reviewing submissions • Procedural matters

16. A note on assessors • For the criteria phase, the funding bodies have appointed panels with broad expertise to develop the criteria; and with numbers in proportion to each panel’s remit • Additional assessors will be appointed in 2013 where the breadth or depth of a panel’s expertise needs to be extended, to undertake the assessment: • Assessors (either academic or user) will play a full and equal role to panel members in developing the sub-profiles (either outputs or impact) • They will be trained and briefed and attend meetings as necessary during the assessment phase