Demonstrating the impact of UK e-Research; a research council perspective

1. Demonstrating the impact of UK e-Research; a research council perspective Drs. Astrid Wissenburg, ESRC Workshop 11: Profiling UK e-Research: Mapping Communities and Measuring Impacts UK e-ScienceALL HANDS MEETING 2008

2. Demonstrating the impact of UK e-Research; a research council perspective • Definitions • Context and drivers • Current approaches - examples • Outstanding issues

3. What is (economic) impact? (1) • “An action or activity has an economic impact when it affects the welfare of consumers, the profits of firms and/or the revenue of government. Economic impacts range from those that are readily quantifiable, in terms of greater wealth, cheaper prices and more revenue, to those less easily quantifiable, such as effects on the environment, public health and quality of life. “ (Treasury) • “improvements for UK society and the economy” (RCUK)

4. What is (economic) impact? (2) • “The concept of 'impact' in the social sciences applies to all sectors: public, private and voluntary. It embraces economic and societal impact in the sense of direct and often quantifiable economic benefits; wider social impacts that will benefit society more generally such as effects on the environment, public health or quality of life; and impacts on government policy, the third sector and professional practice. (….) Economic and societal impact builds on academic impact: developing social science knowledge and methods.” (ESRC)

5. Impact aspects • Academic and non-academic • On all sectors: public, private, voluntary – and general public • Quantifiable and non-quantifiable • Direct and indirect • Policy, practice, products

6. Strategic drivers for impact agenda • Government’s Science and Innovation investment frameworks emphasize the maximisation of public investment in science on the economy • Research councils have been challenged to demonstrate an increase in their economic impact • Major research challenges are interdisciplinary, and require collaboration and co-production to ensure quality research with impact

7. Purposes of identifying and measuring impact • To learn lessons, specifically to understand, improve and support the processes which create impact • To make the case for the science budget • To inform funding decisions

8. Identifying and measuring impact • Growing portfolio of evaluation methodologies to ‘measure’ economic and societal impact: • Across all types of ‘research activities’ – including e-Research • At all levels: national science budget, activity type, funding mode, programmes, individual projects • Across the whole life-cycle of activities, including pre and post

9. Example 1: RCUK Economic Impact study (SQW/PA, 2007) • Series of 18 case studies across training, research and facilities investments from all research councils. • Methodology: stakeholder interviews was supplemented by data and secondary research • Used a classification of impact types: • Development of human capital • Business and commercial • Policy • Quality of life

10. Example 1: RCUK study - impacts • Examples of impacts from case studies include: • Use of scientific facilities by industry: • direct income through selling access • indirect impact through businesses' exploitation of knowledge that is developed by using the facilities. • Software development and licensing revenue • Spin-off companies

11. Example 1: RCUK study - findings • A wide diversity of impacts emerging from research funding, many not part of the original rationale for the specific investment • Not possible to robustly extrapolate from the case study findings to total spend of research councils, partly because the methodology is resource intensive and expensive • Lack of ‘management information’ about outputs from investments, especially post project.

12. Example 2: DIUS study on large scale facilities (2008) • Review of economic impacts relating to the location of large-scale science facilities in the UK • Five facilities covered • Analysis of economic impacts arising from employment, expenditure and knowledge transfer and scientific benefits • Focus on benefit of the location of facilities in the UK as opposed to access to similar facilities abroad.

13. Example 2: DIUS study on large scale facilities (2008) • The major economic impacts arise from: • employment of relatively highly paid staff, most of whom reside close to the facility • the awarding of contracts to UK-based suppliers • Plus: • contributing to local technology clusters but generally a relatively small component • Some examples of transferring knowledge and technologies to suppliers, but no evidence that this is on a substantial scale

14. Example 3: ESRC policy and practice impact evaluations • Case study approach intended to cover different investments and different methodologies, including: • “Payback” method: this method examines the stages of a project throughout its life, and categorises its “paybacks”. • "tracking forward" from research outputs to the way in which these have been incorporated into practice in the public and private sectors. • an analysis of end of award reports to assess the potential for impact, followed by an analysis based on "unobtrusive" or "non-reactive" measures involving web-based searches, and databases of publications • a logic chain, mapping resource inputs to shorter term research outputs and to longer term outcomes.

15. Example 3: ESRC policy and practice impact evaluations • qualitative studies with mixed methods captures the complexities and allows for triangulation • models that anticipate the communication flows between researchers and potential users provide a framework for analysis, but linear models assume a trail of evidence which is seldom present

16. Example 3: ESRC policy and practice impact evaluations – next steps • Economic impact study of 2 Research Centres: • Identify achieved economic impacts (through research, dissemination, networking, etc); • Apply appropriate techniques to calculate values for the economic impact of selected aspects • Present the results within a broader analysis of the Centres’ policy and practice impacts; • Identify lessons for future economic evaluations

17. Methodological issues • How can we address the problems of • Attribution • Traceability • Measurability • Time lags? • How can we move beyond inputs and outputs as proxies for actual impact? • How do we represent the receptiveness of users?

18. Research funders issues • Can/should we establish any robust baseline? • What is the relationship between individual projects to programmes to investment portfolios to the science budget? • What we can do: • More systematic data gathering of output data and ‘impact’ stories, during and post projects • More testing of economic impact valuations • Extract the lessons about good pre-conditions and practice for impact creation

19. E-Research Issues • Are the current approaches applicable to assess the impact of e-Research? • Understand pathways to impact, both direct and indirect, from e-Research • Impact of e-Research on the research process itself and the implications for impact measurement • Can some e-Research tools be sued as part of a portfolio to collect and analyse ‘impact’ information

20. Thank youastrid.wissenburg@esrc.ac.uk