LEARNING through ongoing EVALUATION

1. LEARNING through ongoing EVALUATION Ongoing evaluation of the eight ERDF programs and in major strategic projects. Joint evaluation between the ESF and ERDF of the implementation organisation. Joint university course and reader in Learning Through Ongoing Evaluation at five universities. Learning conferences and seminars in each of the eight regions/structural programmes.

2. Traditional Evaluation vs Ongoing Evaluation Compared to: • Mid term evaluation came too late and was too expensive; minor impact in managing of the programmes. • Quantitative rather than qualitative focus. • Indicatororiented evalutions means heavy focus on activities in the projects (at the expense of the overall objectives). • Limited learning in the projects, in the program context and between stakeholders in national framework (labour market policy and regional growth policy actors).

3. Ongoing evaluation of the organization of implementation: • The evaluation has initially focused on the effectiveness processes of preparation of applications, case handling and prioritisation. During the second half of the programming period, the ongoing evaluation: • focus more on learning and strategic orientation. • and on whether the implementation organisation has managed to assimilate documented experience and steer towards the programme goals better. A main result of this effort has meant that customer focus has risen in both funds and initial conflicts in the implementation organisation have been solved!

4. Ongoing evaluation at the programme level: • An ongoing evaluation and process support structure of the eight Swedish regional fund programmes (ERDF:s) are carried out. • Three teams of evaluators are studying the progress of the programmes, analysing the evaluation of the projects, and evaluating (based on the monitoring system) how the work is progressing at regional level.The main task is to ensure that the project portfolio is well balanced in terms of the programme’s goals, and that the programme steers towards its goals. In the second half of the programming period results and impacts will be in focus. • Learning processes between the funds have been initiated; eg. eight regional learning conferences during 2009.

5. Ongoing evalutation at the project level: • Larger projects are requsted to carry out ongoing evaluation. Larger projects are supposed to set up a plan for ongoing evaluation, call for tender and learn from the ongoing evaluation to support continuous improvements of the projects. • Smaller projects are requested to carry out self-evaluation. Ongoing evaluation has meant a radical change in the perception of evaluation efforts: • Evaluation is now percieved as a means to support the managing of the project. • Ongoing evaluation supports project leader as well as managing authority and other stakeholders with knowledge. • In general ongoing evaluation at the project level has meant a clearer focus on job creation, regional growth and innovation.

6. Billion goes to R & D-related efforts in the Swedish ERDF programs • More than 2 billion SEK is so far granted by ERDF programs, including co-financing more than 4 billion. Corresponds to approximately 40% of the total effort in the programs. • The major project owners are universities but also many others.

7. Preliminary findings from the ongoingevaluation of innovation: 1) Projects might be more innovative if there are other owners of the projects than the scientific community.2) There is a need for clearer plans that lead to commercialization (resources are to often used for traditional research patents that are not passed on to commercialization, peer review articles and so forth).3) Cooperation with business should be increased in projects as well as in monitoring committees and partnerships!4) The initiation of projects should be made in consultation with experts, the degree of innovativeness is focused.5) There is a need for regional innovation strategies that put projects in a regional growth context.

8. 6) There should be an ambition to constantly see how the ERDF projects can interact with other national programs, venture capital fund projects and other parts of the EU programs and funds, for example, how can structural fund projects be carried forward into a Framework project? 7) Also 7th Framework program projects too seldom lead to sharp innovations, but rather "intermediate knowledge outputs” (Technopolis evaluation). Traditional companies are to often participating companies rather than innovative "young" businesses. 8) Ongoing evaluation in projects should be done with a clear focus on innovation, commercialization and business participation. 9) The learning that takes place between the projects and other actors should have a much stronger focus on innovation, commercialization and growth. 10) According to the business society there is to much detailed control rather than a focus on the overall objectives of doing business.

9. There are indications that ERDF programs could do better to support growth if: • innovation projects were carried out in close connection to entrepreneurship programs (cf. Entrepreneurship and Innovation Policies of the authors Anders Lundström, Moa Almerud and Lois Stevenson, 2008). • innovation projects had yet closer linkages to industry, i.e. not necessarily be based on research and scientific results but oriented towards commercialization (cf. the Swedish paradox). • innovation projects are conducted in a strategic context that support sustainable growth and structural change (cf. the Finnish innovation system!).