Research Excellence Framework (REF)

1. Research Excellence Framework (REF) Guidance on submissions

2. REF Timetable

3. Keele proposed Timetable

4. REF Guidance documents Comprehensive information on preparing submissions and on how panels will assess them will be set out in: • ‘Assessment framework and guidance on submissions’ (July 2011) • ‘Panel criteria and working methods’: • July 2011: published in draft form for consultation • Jan 2012: in final form. • Supplementary guidance relating to submissions

5. REF main changes from RAE Inclusion of assessment of ‘non academic’ impact of research Standardise three elements across UoAs Reduce number of UoAs i.e disciplines(from 67 to 36) and main panels (15 to 4) Structured templates (for consistency) Using standardised HESA data Limited use of citation data in some UOAs Measures to promote equality & diversity (CoP) Quality profiles in steps of 1% not 5% Removal of ‘esteem’ as a distinct element 5

6. Content of submission Template page limits in Annex F

7. Individual staff circumstances (REF1b) • Up to four outputs must be listed against each member of staff • This can be reduced without penalty where circumstances have constrained an individuals ability to work productively or produce four outputs during the assessment period • Consistent approach across panels: • Clearly defined circumstances will have a ‘tariff’ based on time ‘absent’ • Complex circumstances requiring judgement will be referred to an expert panel

8. Individual staff circumstances (REF1b)

9. Code of Practice • All HEIs required to develop, document and apply Code of Practice on selecting staff include in the REF submission • fair and transparent, promotes equality and diversity, complying with legislation and avoids discrimination. • Number of outputs that can be reduced without penalty for Early Career Researchers

10. Code of Practice continued • Other clearly defined circumstances e.g. part time working, maternity (maybe) • More complex circumstances e.g. disability, dealt with on a case by case basis.

11. REF weightings across UoAs 15% 65% 20%

12. Impact categories REF = retrospective impact RCs = prospective impact

13. Impact definition – read carefully! • An effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, BEYOND ACADEMIA • It includes an effect, change or benefit to: • The activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding • Of an audience, beneficiary, community, constituency, organisation or individuals • In any geographic location whether locally, regionally, nationally or internationally

14. Impact definition continued Impact ……. • that has taken place during the assessment period (2008 to 2013), and was • underpinned by excellent research (2*) produced by the submitting institution (1993 to 2013) • Includes reduction or prevention of harm, risk cost or other negative effects • Panels provide (not restrictive) guidance on kinds of impact they would anticipate in their UOA, and on appropriate forms of evidence

15. Methodology • Case studies (80%) • Impact template - Submissions will include contextual & strategy info about how the unit has supported and enabled impact between 1st Jan 2008 to 31st July 2013 (20%) • Impacts at any stage of development, but must have taken place during period • NOT future or potential impacts! • NOT dissemination activity without evidence of benefits

16. Number of case studies (80%) Case studies not expected to be representative of spread of research

17. Format of case studies (80%) • Generic template with word limits: • Summary of impact (100 words) • Underpinning research (500 words) - how the research made a ‘material and distinct’ contribution to the impact • References to the research (max of 6) • Details of the impact (750 words) - to explain how research underpinned impact and the nature and extend of impact, i.e. who/what was affected? how were they affected? • Sources to corroborate the impact (max 10) References to independent sources that could verify claims See Annex G for further details

18. Format of Impact template • Statement to describe the units approach to supporting and enabling impact during the assessment period including: • Context • Approach to impact (during 2008-2013) • Strategy and plans for supporting impact • Relationship between the approach and the submitted case studies • Further details in panel criteria, along with ‘Types of Impact’ and ‘Types of Evidence’

19. Attribution and timeframe • Must show that Keele undertook research that made a distinctivecontribution to achieving the claimed impact or benefit • Underpinning research from 1993 to 2013

20. Assessment • Produces an impact sub profile (20%) • Assessed against reach and significance • ‘Reach’ how widely the impacts have been felt • ‘Significance’ how transformative the impacts have been • REF panels provide more info in draft criteria • Impacts cannot be compared across UoAs

21. Lessons learned from pilot Panels • Best case studies make explicit the non academic benefit from research. ‘Brief is best’ • Good case studies showed the link between research and impact and provided supporting evidence • Case studies can get high rating on either ‘reach’ or ‘significance’ (or both) • Engagement isn’t impact • Not convincing to simply state ‘distinguished Professor’ • Universities need to improve their presentation of evidence • Don’t expect panels to follow up references, these are just for verification • Big challenge acquiring supporting evidence (heavy reliance on personal knowledge of senior academics)