Snowball Metrics

1. Snowball Metrics Slides from : Anna Clements, University of St Andrews Lisa Colledge, Elsevier Stephen Conway, University of Oxford Keith Jeffery, euroCRIS Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

2. Snowball Metrics are… • Tried and tested methodologies that are available free-of-charge to the higher education sector • Defined and agreed by Higher Education Institutions to support their strategic decision making • Absolutely clear definitions enable apples-to-apples comparisons • www.snowballmetrics.com Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

3. Sentiments among English research institutions were the prelude to this partnership • Growing recognition of the value of data/metrics to inform and monitor institutions’ research strategies • Dissatisfaction with available tools: multiple custom implementations, incompatibility of data systems • Frustration over the lack of standard metrics available to give sensible, relevant measurements • Frequent data requests from external bodies put pressure on institutions to collect and return data on aspects of performance that are not necessarily best suited to the institutions themselves The origins of Snowball Metrics Imperial College London and Elsevier conducted a study of research information management in England funded by the Joint Information Systems Committee (JISC) • Highlighted recommendations from the study • Institutions and funders should work more collaboratively, and develop stronger relationships with suppliers • An agreed national framework for data and metric standards is needed • Suppliers should participate in the development of data and metric standards ... which led to Snowball Metrics Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

4. Project Partners Defining Snowball Metrics A subset of participants initiated a second phase of work that would address and build on these recommendations Vision: Institutions agree on metrics and methodologies as standards to support strategic decision making by enabling benchmarking across institutions globally Approach: Eight UK higher educations institutions working with a supplier, Elsevier, partnered to reach consensus on a set of metrics. Snowball Metrics reflect universities’ view of the world, not Elsevier’s. Elsevier played a supporting role to help higher education institutions articulate their needs, and to test the feasibility of addressing them Outcome: Agreed and tested Snowball Metrics methodologies have been shared freely in the Recipe Book, so that any organization can adopt the framework Next steps: All project partners are committed to facilitating a global definition of these metrics as standards by incorporating other existing standards into the “recipes”, and working with all suppliers to implement these metrics Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

5. Snowball Metrics Landscape • Metrics are sliced at the institutional and discipline / funder type levels • Metrics need a combination of institutional, proprietary, and public data Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

6. Snowball Metrics shared in Recipe Book Input Metrics Applications Volume Awards Volume Process Metrics Income Volume Market Share Scholarly Output Citation Count h-index Field-Weighted Citation Impact Publications in Top Percentiles Collaboration Output Metrics Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

7. The value of agreeing definitions • Certainty that we are comparing like with like • Identify common framework for comparisons • Institutional organizational structures are different and not suitable to provide context • Agreed a common framework in the UK to benchmark within: HESA Cost Centres • We needed to assign our data to the HESA Cost Centre framework HESA Cost Centre: a grouping of researchers by field that allows meaningful comparisons between different types of data within a discipline HESA: Higher Education Statistics Agency www.hesa.ac.uk Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

8. Specifying Awards Volume Removing ambiguity (extracts from the complete “Recipe”) • Use aggregated values of awards over award lifetime, not the value (to be) spent in any financial year • Date used is the date that the award is entered in the institutional grants system • Include subsequent financial amendments – supplements and reductions • Do not include non-financial amendments such as no-cost extensions http://www.snowballmetrics.com/metrics/ Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)

9. Metrics are tested for feasibility Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

10. Metrics are tested for feasibility Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

11. Metrics are tested for feasibility Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)

12. Our task – to CERIFy the metrics Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier

13. MICE Project: CERIF Architecture

14. CERIF Indicators Segment

15. Physical data Structure Indicators

16. Snowball Metrics Project Partners University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)