1 / 10

Research Evaluation Supported by Automation

Research Evaluation Supported by Automation. Keith G J effery President, euroCRIS k eith.jeffery@stfc.ac.uk. Why Research Evaluation. To protect from erroneous information / knowledge; To demonstrate value for money from public funding; To compare performance of research institutions;

denton
Download Presentation

Research Evaluation Supported by Automation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Evaluation Supported by Automation Keith G Jeffery President, euroCRIS keith.jeffery@stfc.ac.uk euroCRIS Seminar 2010

  2. Why Research Evaluation To protect from erroneous information / knowledge; To demonstrate value for money from public funding; To compare performance of research institutions; To compare judgement of proposal review process in different funding organisations; To promote the research outputs achieved; euroCRIS Seminar 2010

  3. Kinds of Research Evaluation • The Research Grants process • Evaluation of proposals submitted for funding • Evaluation of research outputs from funded projects • The Research Communication process • Evaluation of scholarly publications submitted • The Formal Evaluation • Evaluation of research grant income • Evaluation of research outputs (particularly publications but also products and patents) • Evaluation of contribution(s) to academic communities euroCRIS Seminar 2010

  4. How Research Evaluation • Commonly an intensive manual process involving • The funding organisation(s) defining a template • The research institutions struggling to locate, collate and present the information • Along the way making strategic decisions on whom and what to include and within which category to place the work • The funding organisations setting up evaluation panels which effectively re-peer-review the submissions • Leading to allocation of funding euroCRIS Seminar 2010

  5. Cost of Research Evaluation • According to PA Consulting the cost of UK RAE2008 was > £47m • A recent report for JISC in UK estimates that with automation this can be reduced by at least 25% • According to PA Consulting the cost of UK submission and monitoring of research grants is £85m p.a. • A recent report for JISC estimates that with automation this can be reduced by 33% euroCRIS Seminar 2010

  6. The Automation • The JISC-funded EXRI study recommended CERIF for research information in UK • A group representing universities, research funders, Higher education authority, research managers, charities convened and approved the recommendation of CERIF for research information in UK • The major automation envisaged is CERIF for • Making returns for research evaluation • Managing the research grant process • But there are additional benefits in managing research • Publicity and outreach including to innovators • Research collaborations • Finding reviewers • Strategic decision-making euroCRIS Seminar 2010

  7. The message From the JISC Report: • For any given institution the minimal annualised savings of owning a modern CRIS using CERIF or a CERIF wrapper can be expressed as: • (X*225)+(y*3200) • Where • X is the number of researchers to be submitted to the REF and • Y is the number of grant applications per annum to Research Councils. • Using a typical institution with 1000 researchers and 1000 grant applications this gives the minimal saving as £3.4m per institution per annum • Balanced against the savings is the cost of either a CERIF wrapper (£13,000) or CERIF CRIS (£10,450 - £20,880) per annum • Which, frankly, is ‘lost in the noise’ of the benefit euroCRIS Seminar 2010

  8. What Automation • Many research institutions have a mix of systems: HR, Finance, webpages, institutional repository • Some have a Research Information Management function – commonly by linking together heterogeneous systems • Some have such a function by using a CRIS - and of those many have a CERIF-CRIS • More organisations specify CERIF for their CRIS • ESF, ERC... Several UK universities euroCRIS Seminar 2010

  9. Benefits of Automated Evaluation • Research Institution • Reduced cost • Increased accuracy • Information available for other purposes • Research Funder • Reduced cost • Increased accuracy • Information available for other purposes euroCRIS Seminar 2010

  10. Benefits of Automated Evaluation • A clear transparent, reproducible process for evaluation: • Based on open data that should already exist in a well-managed research institution • Using algorithms not human opinions • Thus providing the public with assurance of research output quality and value for money from research funding euroCRIS Seminar 2010

More Related