Research Information and Analytics at Cambridge

1. Research Information and Analytics at Cambridge Insight over measurement Dr Owen Roberson Research Information Analyst owen.roberson@admin.cam.ac.uk @oroberson ORCiD: 0000-0001-6938-5162 Research Information Office, Academic Division

2. Introduction – my environment Research Office Academics Grant reporting (e.g. Researchfish) Research Strategy Office Administrators/ Co-ordinators Research Information team REF return University Library HESA (return on statistics) Senior Academic Committees DSpace CRIS Grants HR

3. Research Information Landscape • What is Research Information? • Publications • Grants • Conferences • Patents, software, compositions, performances, exhibitions…amongst others. Not BIG DATA but WIDE DATA.

4. What it’s like being a researcher Where is your Research heading? What were your recent publications? Does it have an international dimension? I like doing research! What’s your next project? WILL IT HAVE ANY IMPACT? Are you doing anything interdisciplinary? Where did you publish them? Did it make a spinoff? Where is your grant money? Are they Open Access? Did it have any Impact? How many grants have you applied for so far? Is the data open too? How many grant applications were successful? How did you pay the APC? How many students are you teaching? How are you going to sustain your grant income profile? Was it well received? How many students are you supervising? What did you do with the grant income you received? Did it have any impact? Does it have any IMPACT? How did you cost your overheads?

5. So what makes a good analytics approach? You can generate statistics on all of these things, but be cautious of the tail wagging the dog. • What is your goal as an analytics team? • Once you have an analysis, what can you actually do? (How many carrots and sticks do you have?) • What are the implications of using the sticks?

6. What can we do, What should we do?

7. What Research Information is NOT, and the perils of metricide • What is a KPI in the context of Research? • How sure are you that you’re measuring what you think you’re measuring? • Are you presenting it fairly? • If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?

8. What Research Information is NOT, and the perils of metricide • What is a KPI in the context of Research? • How sure are you that you’re measuring what you think you’re measuring? • Are you presenting it fairly? • If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?

9. What is a KPI in the context of research? There are none that work. This is a game where the rules and goals change continuously. • Donald Schon’s Reith Lecture: “The Loss of the Stable State”, as part of the series Change and Industrial Society, 1970. http://www.bbc.co.uk/programmes/p00h65rn#play

10. What Research Information is NOT, and the perils of metricide • What is a KPI in the context of Research? • How sure are you that you’re measuring what you think you’re measuring? • Are you presenting it fairly? • If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?

11. How sure are you that you’re measuring what you think you’re measuring? • Impactful • REFability • EPSRC Compliancy http://xkcd.com/552/

12. What Research Information is NOT, and the perils of metricide • What is a KPI in the context of Research? • How sure are you that you’re measuring what you think you’re measuring? • Are you presenting it fairly? • If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?

13. What Research Information is NOT, and the perils of metricide http://figshare.com/articles/What_aren_t_you_seeing_Advance_in_visualisation_reserach_/1464961 Once you’ve got a metric, are you presenting it fairly?

14. What Research Information is NOT, and the perils of metricide • What is a KPI in the context of Research? • How does calculating something inappropriate affect the conversation? • How sure are you that you’re measuring what you think you’re measuring? • Are you presenting it fairly? • If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?

15. Scientocentricity, alienation, and bad incentives http://xkcd.com/435/

16. Also… NEVER USE JOURNAL IMPACT FACTORS

17. If you’re going to do it, do it well • Research Information must: • Provide insight • Be explained in its context • Be aware of its limitations • Good places to start:1 • Have a statement about your position re DORA • Be aware that the watchmen are being watched too.. • ____ • 1. Thanks to Jane Tinkler, whose original presentation can be found here.

18. Making it work: enter once, use multiple times http://xkcd.com/927/

19. Research Information Systems @ Cambridge • With apologies to Harry Beck….

20. Core I: Symplectic meets institutional data

21. Core II: Further mapping of internal data to Symplectic

22. Core III: REF – Research Assessment

23. Post REF: Altmetrics and ORCID

24. Post REF: Reporting tools, CV & biosketch generation

25. Closing the gap: Open Access

26. Bridging the gap: Dimensions & rich grant meta data

27. From Integration to Innovation – VIVO and network analysis

28. And what all this means… Social Media Impact and Comms Strategy Genuinely new and useful information researchers want to hear Publication Strategy Academic Workload HEFCE Compliance Grant Capture, RCUK Compliance

29. Network Analysis: finding connections It is possible to explore the connections between members of a defined group and consider not only the direct connections between them (through collaboration or publication), but who their collaborators collaborate with, and what other connections they may have made. Colours indicate University departments.Points or nodes represent researchers.Connections or edges represent an instance of co-authorship. This shows collaborations between REF2014 UoAs; indicating that interdisciplinary research is already a key element of research practice at Cambridge. • When combined with contextual knowledge from research support staff (with specialist area knowledge) we can use this type of insight to: • Provide introductions between researchers with similar interests. • Model which topics lend themselves to extensive or intensive groups. • Identify individuals who may be ready to collaborate on a bid. • Help researchers to navigate an evolving research environment. Roberson, Owen; Wastl, Juergen (2015): Beyond Serendipity: Networks of Research at Cambridge. figshare. http://dx.doi.org/10.6084/m9.figshare.1431829

30. Glossary Altmetric.com – Provider of article-level metrics focussing mainly on social media platforms, reliant on DOIs in the main, to track articles through internet sources. Also covers Wikipedia entries and some policy documents. Digital Science portfolio. CASRAI – Not-for-profit seeking to reduce the administrative burden on academics, by supporting single entry, multiple use of data. Dimensions – Provider of publically available grants data, including abstracts and semantic searches. Medical and US/UK focussed at the moment, but growing constantly. Digital Science portfolio. DOI – Digital Object Identifier – unique string used to identify an online object. Operates like a URL, and is public. Mainly applied to articles in the research space, use will probably grow in the near future. DORA – San Francisco Declaration on Research Assessment: an open declaration from researchers and institutions that commits its signatories to the improvement of research assessment metrics and the avoidance of Journal Level Metrics. ORCiD – a unique identifier for a researcher that operates like a DOI for an individual. ResearchFish – the Company and software used by a number of funders, but crucially by RCUK, to track research outcomes for the grants its participants have funded. Mandatory annual returns. Database uses the grant as its principal component. RIMS – Research Information Management System – a database containing metadata about publications. Key offerings in the field are Symplectic Elements, PURE and ePrints, as well as a host of institution-specific, home-grown solutions. Most solutions use the academic as the principal component. Symplectic – Company that produces the Elements system, the RIMS that we use at Cambridge. Digital Science portfolio. VIVO – Open Source ontology of researchers and their research outputs originally developed at Cornell. Can be driven by a RIMS, used by Melbourne for “Find the Expert”.

31. Further Reading and Links: https://responsiblemetrics.org/about/ http://orcid.org/ http://www.vivoweb.org/ http://casrai.org/ http://figshare.com/articles/What_aren_t_you_seeing_Advance_in_visualisation_reserach_/1464961 http://www.research-information.admin.cam.ac.uk/ http://figshare.com/articles/Beyond_Serendipity_Networks_of_Research_at_Cambridge/1431829 Donald Schon’s Reith Lecture: “The Loss of the Stable State”, as part of the series Change and Industrial Society, 1970. http://www.bbc.co.uk/programmes/p00h65rn#play