Download
1 / 38
380 likes | 518 Views
Impact workshop. Phil Hannaford. VP Research and Knowledge Exchange. Agenda. Introduction Overview HEFC Case Study Template Impact case studies- examples of best practice Next steps. Overview. Purpose of the REF.
E N D
Impact workshop Phil Hannaford VP Research and Knowledge Exchange
Agenda • Introduction • Overview • HEFC Case Study Template • Impact case studies- examples of best practice • Next steps
Purpose of the REF The REF replaces the RAE as the UK-wide framework for assessing research in all disciplines. Its purpose is: • To inform research funding allocations by the four UK HE funding bodies (approximately £2 billion per year) • Provide accountability for public funding of research and demonstrate its benefits • To provide benchmarks and reputational yardsticks
Key changes since the 2008 RAE • Inclusion of assessment of impact • Fewer UOAs/panels, operating more consistently • Strengthened equality and diversity measures • Revised eligibility criteria for staff • Addition of (limited) use of citation data in some UOAs • Removal of ‘esteem’ as a distinct element • Revised approach to ‘environment’ and data collection • Increased ‘user’ input on panels; and an integrated role for additional assessors • Publication of overall quality profiles in 1% steps
The assessment framework: Overview 65% 15% 20%
Impact: Definition for the REF (1) • An effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia • Impact includes an effect, change or benefit to: • The activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding • Of an audience, beneficiary, community, constituency, organisation or individuals • In any geographic location whether locally, regionally, nationally or internationally
Impact: Definition for the REF (2) • Impact includes reduction or prevention of harm, risk, cost or other negative effects • It excludes impacts on research or the advancement of academic knowledge within HE; and impacts on teaching or other activities within the submitting HEI • Other impacts within the HE sector, including teaching or students, are included where they extend significantly beyond the submitting HEI
Impact: Criteria * Each main panel provides a descriptive account of the criteria
Reach– How widely felt it was Significance – How much difference it made to beneficiaries
Impact: Template (REF3a) • The unit’s approach to enabling impact from its research: • Context for the approach • The unit’s approach during 2008-2013 • Strategy and plans for supporting impact • Relationship to the submitted case studies • Provides additional information and context for the case studies, and can take account of particular circumstances that may have constrained a unit’s selection of case studies • To be assessed in terms of the extent to which the unit’s approach is conducive to achieving impact of ‘reach and significance’
Impact: Case studies (REF3b) • In each case study, the impact described must: • Meet the REF definition of impact • Have occurred between 1 Jan 2008 and 31 July 2013 (can be at any stage of maturity) • Be underpinned by excellent research (of at least 2* quality) produced by the submitting unit between 1 January 1993 to 31 December 2013 • Submitted case studies need not be representative of activity across the unit: pick the strongest examples • NB. Its not about the esteem or influence of an individual or unit
Impact: Case studies (REF3b) • Each case study is limited to 4 pages and must: • Describe the underpinning research produced by the submitting unit • Reference one or more key outputs and provide evidence of the quality of the research • Explain how the research made a ‘material and distinct’ contribution to the impact (there are many ways in which this may have taken place) • Explain and provide evidence of the nature and extent of the impact: Who/what was affected? How were they affected? When? • Provide independent sources that could be used to verify claims about the impact (on a sample audit basis)
Impact: other key points Impacts may be at any stage of development or maturity Impacts stay with the institution (unlike publications)- so cannot ‘buy-in’ impacts Can be shared with other insitutions (each has to show their disticntice contribution to the imapct) Impacts must have taken place during the assessment period (not future or potential impacts) Impacts or benefits arising from engaging the public with the submitted unit’s research will be eligible (but not dissemination activity unless there is evidence of its benefits) Impacts arising from public engagement must show that that the engagement activity was at least in part based on the submitted unit’s research AND drew materially and distinctly upon it
Glasgow’s pilot experience • Clinical Medicine (17 case studies) + Earth Systems (4 case studies) • Writing team • Steering Group (SG) of senior academics chaired by VP R&E • Process was: • Trawl for stories • Interviewed researchers
Glasgow’s pilot experience Labour intensive for the staff involved collection and collation of the material we need to submit the iterative nature of the drafting process
Glasgow’s pilot experience External supportive evidence Challenging to engage external contacts: - who? - ‘Sorry they left a few months ago...’ - S – t – r – e – t – c – h – i – n – g Goodwill!
Glasgow’s pilot experience External supportive evidence Challenging to engage external contacts: - who? - ‘Sorry they left a few months ago...’ - S – t – r – e – t – c – h – i – n – g Goodwill! NB. We need to think when to engage with external supporters
Glasgow’s pilot experience Possible types of evidence • Testimonials from named individuals • Press coverage • Guidelines/Documents/Reports • Training materials • Details of conference/invitations to speak • Links to relevant background information • Public engagement – speak to the ‘organiser’ of the event • Details of grants • Publications – highlighted where peer-reviewe
Glasgow’s pilot experience Avoid subjectivity • Striking the balance • Once upon a time...
Glasgow’s pilot experience Reading group • External reading team from user community • Clinical medicine UoA • Users and academics • Extremely helpful • Internal reading group
Glasgow’s pilot experience Selecting our case studies • Pipeline – extra stories • Look into the past • Consider ‘reach’ and ‘significance’ • Pilot panel reports • Refer to panel criteria • Furthest along the pathway • Believable • Choose your strongest
Glasgow’s pilot experience • Our results • Very pleased with the overall result – there were no • surprises • Managed to avoid: • Generalised, vague claims • Excessive publication lists or web references • Lack of coherence • Claiming potential impact • Lack of necessary information
Glasgow’s pilot experience Top four tips... • Watch the template limits • Make it easy for the reader • So what? • Start with the impact
Glasgow’s preparations • Mini-REF - best examples of impact • UoA “Pipelines” created (Colleges/R&E) • Prioritising pipelines (UoA Champions/R&E) • Developing case studies with academics • Horizon scanning
Useful resources • HEFCE REF2014 site www.hefce.ac.uk/research/ref/ • Pilot panel reports and best practice examples www.hefce.ac.uk/research/ref/impact/ • HEFCE REF FAQs www.hefce.ac.uk/research/ref/faq/ • Public Engagement – NCCPE Materials www.publicengagement.ac.uk/how-we-help/event-reports/ref-workshop • Your colleagues (in Aberdeen and outside)
Read the guidance !!!!! • Assessment framework and guidance on submissions • Panel criteria and working methods
Preparing an impact case study • Suggested questions to help clarify ‘impact’: • What user groups outwith academia did you work with? • What was the purpose of the ‘interaction’? • What has been the effect on the users/audience? • Did it change something for them? • How did they benefit from the ‘interaction’? • How did your contribution effect the impact/benefit?
