The Research Excellence Framework Design Research Society Nottingham October 2011

1. The Research Excellence Framework Design Research Society Nottingham October 2011

2. Main Panel D Criteria and Working Methods • Main Panel D covers: • Area Studies • Modern Languages and Linguistics • English Language and Literature • History • Classics • Philosophy • Theology and Religious Studies • Art and Design: History, Practice and Theory • Music, Drama Dance and Performing Arts • Communication, Cultural and Media Studies; Library and Information Management

3. REF Sub-panel 34: Art and Design: History, Practice and TheoryChairProfessor Paul Greenhalgh, Sainsbury Centre for Visual Arts University of East AngliaDeputy ChairAnne Boddington, University of BrightonPanel SecretaryMs Brenda Purkiss, University of CambridgeMembersProfessor Oriana Baddeley, University of the Arts LondonProfessor Naren Barfield, Glasgow School of ArtProfessor Tim Benton, The Open UniversityProfessor Sandy Black, London College of Fashion, University of the Arts LondonProfessor Stephen Boyd DavisDr Christopher Breward Victoria and Albert MuseumProfessor Brendan Cassidy, University of St AndrewsProfessor Rachel Cooper, Lancaster UniversityDr Colin Cruise, Aberystwyth UniversityProfessor Juan Cruz, Liverpool John Moores UniversityProfessor Stephen Dixon, Manchester Metropolitan UniversityDr Beth Harland, Winchester School of Art

4. Members continuedMichael Horsham, Tomato Design ConsultantsProfessor Deborah Howard University of CambridgeProfessor Nigel Llewellyn, TateProfessor Judith Mottram, Nottingham Trent UniversityProfessor Magdelana Odundo, University for the Creative ArtsProfessor Stephen Partridge University of DundeeProfessor James Roddis Sheffield Hallam UniversityProfessor Irit Rogoff Goldsmiths, University of LondonProfessor Emma Rose, Lancaster UniversityProfessor Paul Seawright, University of UlsterProfessor Penny Sparke, Kingston UniversityDr Peter Stewart Courtauld Institute, University of LondonDeyan Sudjic OBE, Design MuseumProfessor Sue Walker, University of ReadingProfessor Evelyn Welch, Queen Mary, University of London

5. REF is not RAE

6. Key Differences Between RAE and REF • Impact introduced. • Increased role of ‘users’. • Fewer UoA’s with more subject range. • Role of Main Panel. • Removal of ‘Esteem’ as a criterion. • Adjustment of Environment as a criterion.

7. Overall quality 15% 20% 65% Outputs Environment Impact ‘rigour, originality and significance’ ‘vitality and sustainability’ ‘reach and significance’ Maximum of 4 outputs per researcher Template and data Template and case studies The overall assessment framework

8. 4* 3* 2* 1* u/c Overall Quality Profile 20% 15% 65% 12 36 42 10 0 Quality Level % of the submission Environment Impact Outputs

9. Main Panel D Criteria and Working Methods • Requests for multiple submissions need to meet the criteria specified in the Guidance on Submissions • Requests are expected in: • Area Studies • Modern Languages • Art and Design • Music, Drama, Dance and Performing Arts • Communication, Culture and Media Studies, Library and Information Management

10. Assessment Criteria: Outputs • Eligible Outputs: • Any type of output embodying research as defined for the REF may be submitted • Sub-panels will not privilege any one kind of output above another • Journal rankings will not be used!

11. Assessment Criteria: Outputs • Eligible Outputs: • The output should be submitted without additional material where that is in itself deemed to constitute sufficient evidence of the research • Additional information for practice-based outputs - 300 words describing the research imperatives, research process and research significance • Portfolio in cases where the research output is ephemeral, is one in a series of inter-connected outputs (eg performances etc) or cannot fully represent its scholarly dimensions through the evidence provided above.

12. Assessment Criteria: Outputs • Co-authored, co-edited and collaborative outputs: • May be listed by more than one author from within a single submitting unit or across submissions from different units • In all cases submissions are required to provide an explanation of the nature and scale of the author’s contribution - not expressed as a % • Panels may judge that significant differences in the quality of the respective contributions should be taken into account in the final grades awarded

13. Assessment Criteria: Outputs • Double-Weighting: • To recognise outputs of extended scale and scope = to 2 outputs. No particular type of output will automatically be double-weighted • Institutions may identify up to 2 outputs per individual author which they consider worthy of double weighting and submit a supporting statement • Panels will assess the claim for double weighting separately from the quality of the output (ie double-weighting does not necessarily result in 2 x 4*) • For each claim, institutions may submit a reserve output, which will only be assessed if the claim for double weighting is not justified.

14. Assessment Criteria: Impact • Definitions for the criteria for assessing impact are: • Reach: The extent and/or diversity of the organisations, communities and/or individuals who have benefitted from the impact. • Significance: The degree to which the impact enriched, influenced, informed or changed the policies, practices, understanding and awareness of organisations, communities and/or individuals.

15. Assessment Criteria: Impact • The Main Panel believes that the impact of research conducted in its disciplines is powerful, pervasive and ubiquitous; challenging imaginations and enriching lives economically, culturally, spiritually and educationally • It has provided, as illustration, a range of areas of impact, to help institutions to think about what case studies in the arts and humanities might look like • These are: civil society, cultural life, economic prosperity, education, policy making, public discourse, public services • There is no expectation that case studies should be classified in this way; indeed case studies may well cross the boundaries of these areas or go well beyond them

16. Assessment Criteria: Impact • Examples of Impacts: • A short list of examples of impact is provided in the panel criteria • These are drawn from lengthy lists put together by sub-panels, which we would like to publish in due course as an aid to the sector

17. Assessment Criteria: Impact • Evidence of Impact • Main Panel D acknowledges that all potential records of evidence might not be available and that the integrity, coherence and clarity of the narrative will be essential to the panels in forming their judgements, but nonetheless key claims made in the narrative should be capable of corroboration • Narratives should articulate the relationship between the underpinning research and the impact as well as the reach and significance of the impact itself • An extensive range of types of evidence that could be used to support case studies is provided in the criteria to assist institutions in compiling their case studies

18. Assessment Criteria: Impact • The Impact Template has four sections and will comprise 20% of the impact sub-profile: • Context • Approach to Impact • Strategies and Plans • Relationship to Case Studies • Main Panel D has explained in its criteria the kinds of information it would like to see under these headings; not exhaustive lists • In particular, it recognises that there is not always a planned, causal link between research and its subsequent impact and that pathways to impact may be diffuse and non-linear.

19. Assessment Criteria: Environment • Definitions for the criteria for assessing environment are: • Vitality: The extent to which the research environment supports a research culture characterised by intellectual vigour, innovation and positive contribution to the discipline. • Sustainability: The extent to which the research environment ensures the future health and well-being of the unit and the discipline.

20. Assessment Criteria: Environment • Data required [REF 4a/b/c] • Data requirements have been reduced since RAE2008 to the following three datasets(by year, for the period 1 August 2008 – 31July 2013): • Doctoral awards • Research income by source • Research income-in-kind • Main Panel D has not asked for any other additional data to be submitted. • This will be considered alongside the information provided in the environment template

21. Assessment Criteria: Environment • Environment template [REF5] (equivalent to RA5) • Headings: Overview; Strategy; People (covering staffing strategy and staff development, and research students); Income, Infrastructure and facilities; collaboration and contribution to the discipline. • Panel Criteria specifies the kinds of information sub-panels would like to see under these headings; these are not exhaustive lists. • Word lengths linked to number of ftes submitted.

22. Assessment Criteria: Working Methods • Interdisciplinary and multidisciplinary research welcomed and treated equally. Sub-panels members have been selected to embrace broad-ranging experience to enable assessment of such work and work that crosses UOA boundaries. • Within Main Panel D, cross-referral will be characterised by dialogue between the relevant SPs. • Cross-referrals to other Main Panels if necessary.

23. Assessment Criteria: Working Methods • Additional assessors (both academic and user) will be appointed to each sub-panel to assist with the assessment phase. • Sub-panels will review institutional Statements of Submission Intentions to identify gaps in expertise or areas where the workload will be significantly heavier than anticipated. • There will be an appointments process which will take due regard of advice received from subject associations and other professional bodies.

24. Assessment Criteria: Working Methods • Main Panel will work with sub-panels to ensure adherence to assessment criteria and consistent application of standards. Details defined in Panel Criteria. • Sub-panels will ensure that submissions are assessed using appropriate expertise: approaches defined in Panel Criteria. • User members and user assessors will contribute significantly to the assessment of impact.

25. Assessment Criteria: Working Methods • Reviewing Outputs • “In every submission, all outputs will be examined with a level of detail sufficient to contribute to the formation of a reliable quality profile for all the outputs in that submission.”

26. Further information • Guidance on submissions (July 2011) • Draft panel criteria and working methods (July 2011) • www.ref.ac.uk • info@ref.ac.uk