Econometric Evaluation of Public Innovation Subsidies: State of the Art

1. Econometric evaluation of public innovation subsidies: state of the art, limitations and future researchDirk CzarnitzkiK.U.Leuven and ZEW MannheimParis, November 9th 2010

2. Introduction • R&D is subject to market failure • External effects • Financial constraints • Governments try to „correct“ market failure using several policies • Governments invest in public science • Intellectual property rights systems • R&D collaborations are exempt from anti-trust policy • Public R&D grants or tax credits for companies • Often preferred treatment for research consortia • Recently especially industry science collaborations

3. Introduction: Direct subsidies • Problem: crowding-out may occur! • Once subsidies are available, companies have an incentive to apply for any project (even the privately profitable ones) as subsidy comes at marginal cost equal to zero. • Subsidies may not only stimulate the projects with high social return. • In the worst case, private funding is simply replaced with public funding. • How can government select projects that were not carried out otherwise? • How to evaluate the „success“ of a policy?

4. An example of self-assessment by companies (259 subsidized German companies in 2001)

5. Aim of quantitative methods of evaluation is the measurement of effects generated by policy interventions on certain target variables; • in innovation context, for example: • impact of R&D subsidies on firms‘ private innovation/R&D expenditure (input) • or on other variables like patent applications etc. (output) • David et al. (2000) review the literature on crowding-out effects and Klette et al. (2000) survey microeconometric studies including output analyzes (like firm growth, firm value, patents etc.) • Critique of early surveys: results varied a lot, possibly because of deficencies of methods that researchers used in the past selection bias (next slide) • Recent survey, Cerulli (2010) surveys more recent literature that used econometric methods for „treatment effect“ estimation. The Evaluation Problem

6. In most cases, one is interested in the average „treatment effect on the treated“ (TT), that is, the difference between the actual observed value of the subsidized firms and the counterfactual situation: „Which average value of R&D expenditure would the treated firms have shown if they had not been treated“ Problem: The counterfactual situation is never observable and has to be estimated! The Evaluation Problem S:= Status of group, 1 = Treatment group; 0 = Non-treated firms Outcome: YT = in case of treatment; YC = counterfactual situation

7. How can we estimate the counterfactual situation? Problem: Those firms receiving a treatment may be different from those that don‘t. Thus: We cannot use a random sample of non-treated without any adjustment. Example: Agencies that fund R&D follow a picking the winner strategy, as they want to maximize the outcome of the funded projects.  Firms that show high R&D in the past, professional R&D management, good success with their other R&D projects will be preferably selected by policy makers.  Subsidy receipt becomes an endogenous variable (depending on the firms characteristics).  Solution: experimental setting, that is, a random assignment of treatments;or the evaluation of policy via treatment effects estimators in non-experimental settings. The Evaluation Problem

8. Evidence on treatment effects estimation • „Buzzwords“: • Econometric Matching • Selection models (also parametric treatment effects models, or control function approaches) • Instrumental variable estimation • (conditional) difference-in-difference estimation • Regression discontinuity design • Most recent studies find positive effects of R&D subsidies on R&D investment • However • Researchers often only observed subsidized yes/no, instead of amount of funding. • Only some studies that use exact amount of funding/year.

9. Real effects or wage effects? • Recentliteraturegoesbeyondthemeretreatmenteffect on thetreatedestimation. • Forinstance, „Goolsbee (1998) critique“ • Onemay find increasedinvestmentbecauseofsubsidy • However: riskthatsubsidyonlyendsup in higherwagesofresearchers • If wage increasedoes not coincidewithproductivityincrease, therewouldbeno „real“ effect on knowledgecreation. • Recentevidencefor Europe • Kris Aerts, Ph.D. Thesis 2008 K.U.Leuven, forFlanders • Pierre Mohnen and Boris Lokshinfor NL  „real“ R&D is also stimulatedbysubsidies.

10. Policy design: large versus small firms • Researchers often find larger treatment effects for smaller firms than for larger firms • This may be a statistical artefact to some extent, though • Often only small sample on really large firms (data limitation) • Proportion of subsidized R&D is much smaller in large firms compared to total R&D budget than in smaller firms • Easier to find a „significant“ effect in small firms than in large firms. • Measurement or specification problem • See e.g. Aerts and Czarnitzki (2006), IWT study.

11. Policy design: size of subsidies • Some evidence that small subsidies are „not useful“ • Gonzales et al. (2006) • However, more research is needed here. • Could also be a measurement problem as subsidy is usually related to total R&D of the firm, and thus it is not surprising that small subsidy has no large effects. • If one applies an estimation technique that is very flexible with regard to functional form assumptions • (GPS method for estimating dose-response functions) • …we can learn about about potential crowding out effects in different areas of the subsidy distribution

12. A dose response function

13. A dose response function

14. Estimated elasticities

15. Policy design: e.g. collaborative research Often preferential treatment of R&D consortia, especially industry science collaborations External effects can be internalized within consortia by collaboration Duplicate research avoided Participants can benefit from bundling knowledge and realizing economies of scope (knowledge from a collaborative project can be used for other projects) Thus, there may be a „money effect“ and a „spillover effect“.

16. Evidence from German policyDivision of collaborative research grants by type of research consortia Source: PROFI database from Germany’s Federal Ministry of Education and Research; own calculations.

17. Policy design: e.g. collaborative research • Evidence that spill-over effects are present and that treatment effect of collaborative research is larger than for „individual subsidies“ • Also: spill-over effects larger for collaboration with science • Projects more basic, i.e. more generic in terms of knowledge creation? • Leading to higher economies of scope? • Branstetter and Sakakibara (2002), Czarnitzki and Fier (2003), Czarnitzki, Ebersberger, Fier (2007), Czarnitzki (2009).

18. Policy design: type of R&D • What type of R&D is actually funded by governments in the business sector? • Is it mainly basic research? • Or rather applied research and technological development? • Market failure may be larger for basic research than for other types • Basic research further away from market • Much higher uncertainty about oucomes and industrial applications • Does the agency behave similar to a bank? That is, also prefer less risky projects when making a grant decision?

19. Policy design: type of R&D • Czarnitzki, Hottenrott, Thorwarth (2010) find indeed that firms suffer more from financial constraints with regard to „Research“ than for „Development“ • Also: firms that receive subsidies for basic research show no sensitivity to financial constraints, but non-subsidized do. Grant rate of submitted project proposal by type in Flanders Note: The data were kindly provided by IWT Flanders (own calculations).

20. Policy Mix: Subsidies versus R&D tax credits • How should the government decide on which instrument to use? • Not so much evidence! • Berube and Mohnen (2009) for Canada: among R&D tax credit recipients, firms that receive direct subsidies invest additional funds. • Takalo, Tanayama, Toivanen (2009), structual model on application decision of the firm, grant decision of agency (yes/no and subsidy rate), investment decision of firm • Model allows simulating absence of subsidies, or „tax credits only“ versus „direct grants only“ • Similar effects of subsidies and tax credits

21. Distributional effects Czarnitzki and Ebersberger (2010) apply a standard treatment effects estimation, but then use the results to derive a Lorenz curve of R&D concentration for Germany and Finland (Why Germany and Finland? Only direct grants, but no R&D tax credits available) R&D concentration is lower in actual situation (policy regime with direct subsidies) than in counterfactual situation (absence of any policy)!

22. Distributional effects Lorenz-curve for the distribution of R&D personnel (Germany) Lorenz-curve for the distribution of R&D personnel (Finland)

23. Going beyond „treatment on the treated“ on innovation INPUT • It could be of interest whether firms that are currently not benefitting from subsidies would invest more into R&D if they would receive subsidies • „Treatment on the Untreated“ • Some recent evidence: cross-country comparison Spain, Belgium, Germany, Luxembourg, South Africa (Czarnitzki and Lopes Bento, 2010) • However, also evidence that currently funded companies do not invest more than actually non-funded firms would invest if they received subsidies. • More research is needed here! Our study suffers from severe data limitations. • One could also estimate which firms out of non-recipients would invest most if they would get subsidies • Ongoing research…..

24. Going beyond „treatment on the treated“  innovation OUTPUT • Does subsidized R&D leads to more innovation output eventually • Subsidized projects may fail more frequently than others. So, more patents? Higher sales with new products? • After treatment effects estimation, total R&D can be decomposed into 2 components • The R&D that the firm would have conducted anyway • and R&D that was induced by subsidy (subsidy + additionally triggered R&D) • Some evidence that both components of R&D have a positive impact on patents and new product sales (Czarnitzki and Hussinger, 2004, Czarnitzki and Licht, 2006, Hussinger, 2008). • However: subsidized component‘s productivity is slightly lower then pure privately financed R&D. (consistent with neoclassical theory) • Limitations: Timing of output relative to input?

25. Conclusions and discussion • Many research questions have been addressed. • Yet, the evidence for actual policy making is still limited • Questions: • Young Innovative Companies (YICs), see Veugelers (2009), Schneider and Veugelers (2010) • What is the „optimal“ policy in terms of size of tax credit or mix with direct R&D grants? • What is a superior design of a funding instrument, e.g.: • Application and grant decision versus: • 2-stage process as in U.S. SBIR program: 1st stage conceptual feasibility study (small amount of subsidy), 2nd stage large subsidy (only 30% of 1st stage participants survive, on average). • Sound cost-benefit analysis: treatment effects vs. cost for society.