1 / 15

The RAE is Dead!

The RAE is Dead!. Long live the REF!. The Changes. The RAE looks at three main areas: Outputs Environment Esteem We are used evaluations of Environment and Esteem being “ informed ” by metrics The intention is that under the REF all three areas will be metric determined .

yardley
Download Presentation

The RAE is Dead!

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The RAE is Dead! Long live the REF!

  2. The Changes. • The RAE looks at three main areas: • Outputs • Environment • Esteem • We are used evaluations of Environment and Esteem being “informed” by metrics • The intention is that under the REF all three areas will be metric determined. • This will require the introduction of bibliometrics into the REF.

  3. The Three Most Important Words • You might guess that the three most important words in the REF were: Research Evaluation Framework Unfortunately, you would be wrong!

  4. The Three Most Important Words • Without any doubt the three most important words in the REF are: We Don’t Know! The worrying thing here is that, at the moment, “WE” includes HEFCE!

  5. Surely HEFCE aren’t that stupid? • HEFCE have conducted a pilot study into gathering bibliometric data. • The pilot was to run concurrently with the 2008RAE to provide a basis for comparison. • Unfortunately this proved to be impossible – and so no benchmark is available!

  6. The HEFCE Bibliometric Pilot Study • Involved 16 universities • Covered 35 UoA’s – good coverage in; Science, Computing, Engineering. Very little social science or arts coverage. • The candidates were asked to provide bibliometrics – including citation data on all staff in the UoA which were submitted to RAE2008. • The candidates were asked to provide the information as quickly as possible, consistent with quality checking.

  7. The HEFCE Bibliometric Pilot Study • The “best” university managed to comply in 3 days! • One university took 150 days! – using an undisclosed, but substantial, number of staff. • In two years time we could be asked to do this on an annual basis!

  8. Lessons Learnt from the Pilot. • Difficulties with older or existing information. • The process must be on-going and not just pre-submission. • Data cleansing is very time consuming – formats, mistakes, missing fields etc. • Difficulties linking publications to staff and staff to UoA’s (HESA id). • In some subjects the coverage simply is not there – i.e. the information does not exist!

  9. Does this slow HEFCE down? • Not a bit of it! • Autumn 2009 – consult on main features of REF • May 2010 – outcome of consultation, phasing in timetable, main operational features in place. • 2011-12 – metrics begin to inform funding. • 2012 – submissions to 2013 ‘light touch’ peer review process. • 2013 – undertake full assessment using ‘light touch’ peer review and metrics • 2014 – HEFCE research funding for all subjects driven by REF from this time on.

  10. So, what should we do? • JMU Working Party set up. This includes; • UoA co-ordinators from science/technical, social science and arts subject areas. • Colleagues from Computer Services and the Library. • Representatives from Research Office. • Decided that we must plan for the “worst case” scenario. That is, we will be required: • to return bibliometric data on ALL publications, • by ALL academic staff, • referenced back to UoA’s, • Over a 10 year period, • annually!

  11. Introducing...Sympletic • Sympletic is a publications management system developed in collaboration with Imperial College. • What Sympletic does: • We supply a list of names, aliases, email addresses, affiliations and UoA data. • Sympletic trawls databases looking for publications by these individuals. • If it finds a candidate publication it emails the person concerned and asks them to confirm if the paper is theirs.

  12. Introducing...Sympletic • On confirmation... • Sympletic will download all information regarding that publication. • Sympletic will maintain this information in an updated condition – particularly citation data. • Staff will be able to examine their own record and notify Simpletic of any errors or omissions.

  13. Sympletic • The plus points: • Once set up, the system is self maintaining. • Producing up to date reports at, multiple levels, • University • Faculty • School • Research Group • Individual • Can automatically update websites • Resource for REF, CV’s, Grant Applications, Corporate Publications etc. • Sympletic “talks” to Oracle and Oracle HR.

  14. Sympletic • The minus points: • We need a definite staff list – referenced to UoA’s • Staff engagement • Personal benefits • Managerial benefits • Issues with coverage – WoS and PubMed. Other databases? • Does not solve the problem of the data simply not existing.

  15. Sympletic – Action Points • Software Demonstration • If used the timetable would be: • Spring 2009 – Pilot • Sept 2009 – roll out • March 2010 – full deployment. • Management input needed to promote staff engagement by adopting Sympletic for other processes: • Profs/Readers • Website maintenance • Validation/Accreditation documents • PDPR

More Related