Main Panel B

1. Main Panel B Chair: Professor Dame Ann Dowling Sub-panel Chairs: Panel Advisers: SP07: Professor David Price Dr Karen Ness SP08: Professor Richard Catlow Ms Lesley Dinsdale SP09: Professor Brian Foster Panel Secretary: SP10: Professor John Toland Simon Kerridge SP11: Professor Steve Furber SP12: Professor John Clarkson SP13: Professor Steve Williamson SP14: Professor Roger Falconer SP15: Professor Philip Nelson

2. Main Panel B Criteria and Working Methods Format of document • Section 1: Submissions and units of assessment • Section 2: Assessment criteria: outputs • Section 3: Assessment criteria: impact • Section 4: Assessment criteria: environment • Section 5: Working methods

3. Submissions and UOAs • UOAs 7 – 15, covering physical & mathematical sciences and engineering • Encourage use of research groups to structure submissions • Eligibility for multiple submission requests depends on nature of disciplines covered: • Expected for UOAs 12 (Aeronautical, Mechanical, Chemical & Manufacturing Engineering) and 13 (Electrical & Electronic Engineering, Metallurgy & Materials) • Requests must meet criteria set out in Guidance on Submissions (GOS), para 50 Main Panel B CWM: Submissions and Units of Assessment

4. Assessment Criteria: Outputs Originality, significance and rigour • If appropriate to output type, may consider editorial and refereeing standards as part of the indication of rigour • For some UOAs, citation information will be used as part of the indication of academic significance Types of research output • All types of research output eligible for submission but HEIs should be mindful that purpose is to assess original research; for reviews will assess only original research or new insights reported Main Panel B CWM: Outputs Assessment Criteria

5. Assessment Criteria: Outputs Co-authored outputs • A co-authored output may not be listed against more than one member of staff returned within the same submission (may be listed in different submissions, either same or different HEIs) • Sub-panel9 only • For outputs with 5 or more co-authors, requires information on co-authorship (maximum 100 words) • Used to assess whether significant contribution • Then assess output as a whole • All other sub-panels • Assume co-authors have made substantial contribution (could request audit information); will ignore any information provided • Assess output as a whole Main Panel B CWM: Outputs Assessment Criteria

6. Assessment Criteria: Outputs Double weighted outputs • Allowed, but expected to occur only rarely for Main Panel B disciplines • Must be requested by submitting institution (maximum 100 words) • Assess request for double weighting, then assess quality • No reserves Main Panel B CWM: Outputs Assessment Criteria

7. Assessment Criteria: Outputs Additional information on outputs • Clarification of research process and/or content (maximum 300 words) • For non-text or practice-based outputs: description of research process and content if not evident within output • For reviews: identification of original research or new insights reported • Additional factual information (maximum 100 words) • Sub-panels 11, 12, 13, 14, 15, only: factual information about significance of outputs not evident in outputs eg how gained recognition, led to further developments or been applied; opinion will be ignored • Not required for sub-panels 7, 8, 9, 10: will ignore any information provided in this category Main Panel B CWM: Outputs Assessment Criteria

8. Assessment Criteria: Outputs Additional information on outputs (continued) • Clarification of content of outputs with significant material in common with output published before 1 January 2008 (GOS para 113) • General principle that panels will assess the original content or new insights reported in the output. • Recognition that some would not be considered by the panels as representing the published version of the work, for example, preliminary findings disseminated to a limited audience, technical reports, or some forms of conference contribution. These will be assessed in full. • HEIs provide details of outputs, and, where appropriate, information on how far the original work has been revised to incorporate new findings, including identification of the new research or new insights reported, additional to the earlier published output (maximum 300 words) Main Panel B CWM: Outputs Assessment Criteria

9. Assessment Criteria: Outputs Citation data • Sub-panels 7, 8, 9, and 11 will make use of citation data, where it is available, as part of the indication of academic significance, as follows: • Data will be provided by the REF team; in addition sub-panel 11 will use Google Scholar; no other sources of bibiometric analysis will be used • Will not be a primary tool in assessment • Absence of citation data will not automatically be taken to mean absence of significance • Mindful that citations can be an unreliable indicator – some forms of output, equalities issues • Sub-panels 10, 12, 13, 14 and 15 will not receive or make use of citation data, or any other sources of bibliometric analysis Main Panel B CWM: Outputs Assessment Criteria

10. Assessment criteria: Impact Range of impacts • The panel expects to see impact across a wide range of types and will consider all equitably in the assessment of submitted case studies • Case studies may describe more than one type of impact • Impacts may be manifest in a wide variety of ways and the list of types provided is only indicative; the panels will consider any impact that meets the general definition in the GOS

11. Assessment criteria: Impact Evidence of Impacts • An indicative list of types of evidence is provided but this is not exhaustive; sub-panels will consider any appropriate qualitative or quantitative evidence that is independently verifiable • Highly confidential or sensitive impacts - consultation proposal on how to deal with these • Impacts arising from public engagement activity – importance of link to underpinning research and evidence of reach beyond just dissemination

12. Assessment criteria: Impact Underpinning research • Submitting unit must demonstrate that it made a significant contribution to the underpinning research • Importance of demonstrating how the research led to, underpinned or contributed to the impact • Threshold quality of research defined as 2* • Up to six key references to the underpinning research may be listed in the case study; up to three of these that best indicate the quality should be highlighted

13. Assessment Criteria: Impact Reach and Significance • Reach is the extent and breadth of the beneficiaries of the impact • Significance is the degree to which the impact has enabled, enriched, influenced, informed or changed the products, services, performance, practices, policies or understanding of commerce, industry or other organisations, governments, communities or individuals • Impacts may be submitted at any stage of maturity but early stage or interim impacts may score less highly than more mature impacts

14. Assessment criteria: Impact Impact template • Main Panel B’s CWM give further information on the range of information it expects to see in each section • Sections b. (Approach to impact) and d. (Strategies and plans for supporting impact) will be assessed holistically. • Sections a. (Context) and d. (Relationship between the approach and the submitted case studies) are for information and will not be assessed

15. Assessment Criteria: Environment • Criteria are • Vitality: extent to which a unit provides an encouraging environment for research, has an effective strategy, is engaged with the national and international research and user communities, and is able to attract excellent postgraduate and postdoctoral researchers. • Sustainability: consideration of leadership, vision for the future and investment in people and infrastructure and, where appropriate for the subject area, the extent to which activity is supported by a portfolio of research funding. • Will interpret ‘environment’ as relating to both the research environment within the submitting unit and its participation in, and contribution to, its subject discipline and academic community Main Panel B CWM: Environment Assessment Criteria

16. Assessment Criteria: Environment Environment template (REF5) • Headings – and weightings in assessment • Overview information only • Strategy 20% • People 30%(staff and students) • Income, infrastructure & facilities 30% • Collaboration & contribution to 20%the discipline • Environment quantitative data (REF 4a/b/c) assessed under • People (doctoral awards) • Income (research income including in-kind) • Panel criteria specifies the kinds of information sub-panels would wish to see under these headings Main Panel B CWM: Environment Assessment Criteria

17. Assessment Criteria: Environment Environment template (REF5) - continued • Some sub-panels are requesting additional quantitative information to be included in REF5 to be considered under specific headings: • In relation to People: • Sub-panels 7, 8, 9 and 11: FTE number of research assistants on staff census date (31 October 2013) • Sub-panels 8 and 11: FTE number of registered postgraduate research students on 31 July for each year of the assessment period • In relation to Income, infrastructure & facilities: • Sub-panel 9 only: additional information on usage of major national and international facilities not supported by the Research Councils Main Panel B CWM: Environment Assessment Criteria

18. Assessment Criteria: Working Methods • Interdisciplinary and multidisciplinary research welcomed and treated equally. Sub-panels members have been selected to embrace broad-ranging experience to enable assessment of such work and work that crosses UOA boundaries. • Additional assessors (both academic and user) will be appointed to each sub-panel to assist with the assessment phase where required. Sub-panel requirements for assessors in will be informed by HEI submission intentions.

19. Assessment Criteria: Working Methods • Main Panel will work with sub-panels to ensure adherence to assessment criteria and consistent application of standards. Details given in Panel Criteria • Sub-panels will ensure that submissions are assessed using appropriate expertise: approaches defined in Panel Criteria • User members and user assessors will contribute significantly to the assessment of impact and may also assess outputs where appropriate • All outputs will be examined in a sufficient level of detail sufficient to develop a robust quality profile

20. Further information • From our website: www.ref.ac.uk • From your nominated institutional contact or data contact • Or email info@ref.ac.uk