Innovation policies and their evaluation Some lessons from the Tuscan experience

1. Innovation policies and their evaluation Some lessons from the Tuscan experience Marco Mariani (IRPET – Tuscany’s Regional Institute for Economic Planning) SEE PROJECT - THEMATIC WORKSHOP 3: Evaluating design and innovation policies Florence, 10-11 May 2010

2. Viewpoints on innovation policies and their evaluation Two different approaches to innovation Two different conceptions of innovation policies Different evaluation objectives and tools

3. The rationale of support policies • Approach A: innovation process at firm level  market failure because of insufficient individual incentives to R&D  public support to single enterprises (e.g. firm subsidies, tax benefits) • Approach B: innovation resulting from webs of relationships  systemic failures  public support to networking and university-industry linkages (e.g. support to joint network projects)

4. Consequent evaluation goals and methodology • Approach A: has the firm performed additional R&D, has it become more innovative or competitive solely because of public incentives? Methodology: the performance of the aided enterprise is compared to that of a non-aided “twin”, according to a counterfactual, quasi-experimental paradigm (e.g. differences in differences; matching) • Approach B: has a network of new innovation-generative relationships come into being, as a result of public support? And is this network able to make firms more competitive? Methodology: networks elicited by policies are analysed in their structure and performance by means of social network analysis tools; then they are compared to pre-existing networks (if any)

5. Tuscan examples of Approach A Theory: mainstream micro-economic theory Main goals: impact of public aid on innovation (R&D expenses, patents, etc.), sales, productivity, # employees, labour cost, mark-up/pricing power… Methods: propensity-score matching Mattei and Mauro (2007) Specific regional programme for SMEs 2000-2006: 266 treated, 721 non-treated Merito et al (2010) National programme- Special fund for applied research 1998-2004: 185 treated, 925 non-treated Irpet Eval. Group (2010) 2 regional programmes 2000-2006: ongoing project

6. Approach A: Evaluation questions (Q) and results (R) Q: Has the programme had any impact on the expansion of sales and number of employees of beneficiary enterprises? R: YES. The increase in the number of employees occurs in the long run (M et al) The impact on sales and employees is stronger for enterprises that benefited of repayable loans; weaker for enterprises that benefited of non-repayable subsidies (M&M). Q: Has the programme had any impact on the amount of skilled labour? R: YES in the short run, and in particular for SMEs (M et al) Q: Has the programme had any impact on the labour productivity of beneficiary enterprises? R: NO (M et al) Q: Has the programme had any impact on the patenting activity of beneficiary enterprises? R: Scarcely, and only in the short run. The policy has probably displaced existing R&D investments, instead of stimulating new ones (M et al)

7. Tuscan examples of Approach B Theory: micro-foundations of innovative systems; agent-based perspective to innovation processes Main goals: identification of emergent structures; contribution to the formation of innovation clusters Methods: Social network analysis tools Bellandi and Caloffi (2010) All network programmes 2000-2006: 122 projects, 908 agents Caloffi and Mariani (2010) All network programmes 2000-2006 on advanced mechanics: 30 projects, 289 agents Russo and Rossi (2009) RPIA 2002-2004: 36/14 projects, 409/203 agents • The programme’s basic idea: networks are conductive of innovation, especially those involving heterogeneous agents (SMEs, universities, service providers) • Broad sectoral-technological focus (no explicit focus on design activities)

8. Approach B: Evaluation questions (Q) and results (R) Q: To what extent has the programme supported pre-existing networks of relationships or mobilized new ones? R: YES in advanced mechanics, optronics, medical devices (R&R, B&C, C&M); NO in biotech and traditional sectors (B&C) Q: Has the programme succeeded in promoting the creation of well-performing (turnover, patents…) networks capable of integrating heterogeneous competencies and to foster systemic effects in the regional economy? R: YES, but in limited technological areas (R&R; B&C) Q: What are the features of the emergent network structure resulting from the policy interventions? R: Some desirable agents and links are often missing (R&R, B&C; C&M) Q: Has the policy been able to mobilize a wide range of co-localised competencies and to favour the formation of an innovation cluster? R: To a very limited extent (C&M)

9. Lines for future research How can existing methodologies for the evaluation of micro/meso-impact be re-thought o re-adapted in the case of public policies promoting design?

10. Some references on the Tuscan/Italian case Bellandi, M., Caloffi, A. (2010), “An analysis of regional policies promoting networks for innovation”, European Planning Studies, 18(1), pp. 67-82. Irpet Evaluation Group- Bonaccorsi A., Brancati R., Mariani M., Mealli F., Trivellato U. (2010), “L’impatto degli incentivi regionali alla R&S. Il caso della Toscana”, in progress-forthcoming. Bondonio D., Martini A. (2001), “Using Event History Analysis to Evaluate the Impact of Investment Subsidies Targeted to Youth-Owned Firms”, Atti del convegno SIS 2001 “Processi e metodi statistici di valutazione”, Roma Tor Vergata, giugno, pp. 277-282. Caloffi A., Mariani M. (2010), “Designing policy support to innovation clusters. An overlapping network approach”, paper submitted for presentation to the EUNIP Conference 2010, Faculty of Economics, Universitat Rovira i Virgili, Reus, 9-11 June. Mattei A., Mauro V. (2007), “Analisi e valutazione delle politiche di sostegno alle imprese artigiane in Toscana”, IRPET, febbraio. Russo, M., Rossi, F. (2009), “Cooperation partnerships and innovation. A complex systems perspective to the design, management and evaluation of a EU regional innovation policy programme”, Evaluation, 15(1), pp. 75-100. Merito M., Giannangeli S., Bonaccorsi A., (2010), “Do Incentives to Industrial R&D Enhance Research Productivity and Firm Growth? Evidence from the Italian Case”, International Journal of Technology Management , 49(1-3), pp. 25-48.