Evaluation of Cluster Development Programs: What Do We Know? Carlo Pietrobelli carlop@iadb

1. Evaluation of Cluster Development Programs: What Do We Know?Carlo Pietrobellicarlop@iadb.org Changing Paradigm of Cluster Development: Learning from Global Experiences New Delhi, 20th – 22nd February, 2014

2. Content of typical IDB-supported cluster program Objective: Promote the competitiveness of firms in clusters by providing sector-specific public/club goods Activities: • Cluster mapping and prioritization (often underlying objective is to reduce inter-regional development gaps by fostering local economic development) • Strengthen local governance(actors’ awareness and mobilization) • Joint participatory design of Plans to Improve Competitiveness (diagnostics, business identification, SWOT, benchmarking, action plan) • Action Plan may include: Training and capacity building to promote inter-firm cooperation, Creation of public or club goods, other initiatives consistent with strategy e.g. TA, R&D, export promotion, marketing, logistics, etc.

3. The IDB support to clusters in LAC since 2000s has been remarkable WARNING: Many other programs have been developed by each country, notably in Brazil, where clusters have represented a common approach to local industrial policy for over a decade now.

4. Following Questions immediately arise: • Did the program have an impact on the productivity of firms in the cluster? What is the size of the impact? • Important: We are talking about causality (attribution to the program) • What was the process of having an impact? • Did networks and better coordination have an effect? • Consider the costs and net benefits of the intervention • Did the program have an impact on the productivity of the firms-cluster-region? (general equilibrium problem)

5. Why evaluations Solid evaluations crucial because: • resources are public and limited, their funding needs to be justified • recipients need to be accountable • most importantly, we need to learn how to improve program effectiveness and contribute to improving livelihoods through enterprise and socio-economic development.

6. … but evaluations of CDPs are difficult CDP are complex, they are mainly indirect and facilitative, system-oriented. They do not consist of a standardized “treatment”, but usually include a menu of specific interventions. Effects require long periods of time to fully manifest (3-5 years) This implies strategic and operational problems. Effects on linkages require relational data. Inter-firm linkages are an outcome of interest in itself and information about them is hard to collect, requiring specific techniques of analysis (SNA). Spillover effects need to be measured. Intra-cluster spillovers make difficult to construct appropriate counterfactuals. Spillover process needs to be modeled. Labor mobility and geographical proximity have been used to measure spillovers.

7. The IDB is devoting substantial efforts to improve the evaluation of CDP - A Toolkit • Econometric work on firm-level performance with Census data, control groups and Propensity Score Matching (PSM) • Analysis of linkages, networks, and their influence on firms and clusters performance through Social Network Analysis • Case studies to explore processes of change and policy effects • Pietrobelli, Maffioli, Stucchi (Eds.), 2014, The Evaluation of Cluster Development Programs, Washington DC: IDB. • Giuliani E., Maffioli A., Pacheco M., Pietrobelli C., Stucchi R., 2013, “Evaluating the Impact of Cluster Development Programs”, IDB TN-551 July. http://www.iadb.org/document.cfm?id=37925857&pubDetail=1&wt_docType=Technical Notes&wt_docnum=37925857&wt_language=en&wt_department=IFD/CTI&lang=en 7

8. Someresults IDB Cluster programs in Minas Gerais and Sao Paulo (Brazil): Econometric evaluation withPropensity Score Matching (PSM, define a groupfofirmswithsameprobability) and Difference-in-Differences (DD, before-after and with-without). They combined analysis of Cluster firms data with Census data, building control groups with DD and PSM • Positive direct and significant effects on: • Employment: about 20% increase in 3-5 years; • Probability to export: about +5% per year relative to the original proportion of exporting firms; • Export levels: between50% and 80% for each exporter for beneficiary firms. Effects persist and grow overtime. • Indirect effects on firms localized in the area of influence of clusters, especially on the probability to export and less on export levels. In Pietrobelli, Maffioli, Stucchi (Eds.), 2014, Cluster programs Evaluation, Washington DC, BID.

9. Some results • Programas Asociativos en Chile – PROFOS(Rivas, 2010) • For each US$ invested in these programs, additional US$ 2.4 were generated. • Larger fiscal revenues: 3.2 pesos of additional VAT revenue for each peso invested in the program (in net present values) • Better responses to crises. Between 1996 and 1999 (Asian crisis) non-beneficiary firms decreased their sales at 2.1% per year; those firms participating in PROFOS assessed in 1996 grew at 12.9% per year due to the changes in behaviour induced by the program(Departamento Economía de la Universidad de Chile, 2003).

10. SNA Methodologies Applied to CDP Evaluation: The Córdoba Electronics cluster in Argentina • A CDP in electronics industry in Córdoba, Argentina, during 2003-2007, co-funded by MIF/IDB. • Objectives: stronger local linkages and cooperation; easier local firms’ access to new production technology and organizational innovations; Better access to new markets. • Information network (IN, to transfer business information), and the Collaboration network (CN, i.e. projects with joint collaboration). • From 2005 to 2012 networks became less dense and more hierarchical. Central enterprises ensured network connectivity and created links between treated and untreated firms. • Local firms plan their linkages and select those from which they believe they can obtain tangible benefits. • CDP strengthened and created new technology-transfer ties between the electronics firms in Córdoba and other institutions.

11. Case studies also offer advantages • They allow the in-depth study of selected issues and cases. • They offer an intuitive understanding of the results. • They provide critical insights into beneficiaries’ perspectives. • They may permit consistency checks and offer feedback to help interpret the findings of the quantitative analyses. • Useful when quantitative models risk to remain a black box and miss the core process of cluster development. • IDB financed many case studies(e.g. Argentina (Río Negro): Fruits-tourism-high tech, Brasil (São Paulo): Shoes and building material, Chile: Fresh fruit (grapes), avocado & higher education, Uruguay: Tourism & blueberries). Forthcoming Casaburi and Pittaluga, 2014.

12. Conclusions • Microeconomic interventions to promote cluster development address a complex social phenomenon based on multi-actor coordination • This complexity demands the combination of multiple approaches for its proper evaluation • Recent impact evaluations using limited but rigorous econometric methodologies show positive and significant results • SNA promises to shed light on the complex workings of inter-firm networks • Case studies remain a crucial way to guide policy-makers and implementing agencies on the specifics of the how and what works better in different clusters in very diverse contexts