Planning and running a regional evaluation system: the Madrid experience

1. Planning and running a regional evaluation system: the Madrid experience Beatriz Presmanes Coordinadora de Programas

2. The Madrid R&D System R&D expenditure 1,7% of GDP 50.000 R&D workers 2% of working population 15 universities More than 50 research centers More than 20 interface organizations 32% of public research centres 31% of international publications

3. REGIONAL PLAN FOR SCIENTIFIC RESEARCH AND TECHNOLOGICAL INNOVATION Regional Framework for R&D&I Implementing a territorial Intelligence System Gaining profit for regional intellectual capital value Creating and capturing knowledge Sharing and linking interest

4. building links economical support Creating knowledge Economical value

5. EVALUATION SCHEME Three levels for evaluation: REGION Regional Innovation System Mapping science and technology capabilities Measuring science and technology production PROGRAMME Regional R&D Programme PRICIT Foresight activity link Technology Watch link INDIVIDUAL GRANT Project evaluation

6. Gathering information trough the madrimasd network in collaboration with academic partners Long term project funded by CM under agreement with relevant research groups Three projects: Regional Innovation System Mapping science and technology capabilities Measuring science and technology production A regional dynamic vision of our capabilities constraints and trends www.madrimasd.org/indicadores/

7. Prof. M. Buesa et al UCM Evaluation of the Regional Innovation System –First step analysing science, technology and innovation indicatorswww.madrimasd.org/informacion/documentos

8. Evaluation of the Regional Innovation System-second step Developping a regional innovation index (IAIF) to determine regional innovation capacity and its temporal evolution 31 adjusted variables analysed, 5 year period, four group of variables: regional and productive environment University Public administration enterprises Prof. M. Buesa et al UCM

9. Evaluation of the Regional Innovation System: some results Innovative regional capacity according regional innovation index IAIF

10. "Intellectual Capital Project" Main targets: To develop a versatile tool adaptable to the needs and characteristics of R&D institutions elaborate dynamic indicators concerning intellectual capital create an operative tool useful through the madri+d platform Prof. E. Bueno and P. Morcillo UAM

11. HUMAN CAPITAL STRUCTURAL CAPITAL RELATIONAL CAPITAL Intellectual Capital Project: Intelect model An integrated an evolutive model Present Future Prof. E. Bueno and P. Morcillo UAM

12. Intellectual Capital Project: the architecture of the map NODES IDENTIFICATION KEY WORDS RESEARCH & TECHNOLOGICAL PRODUCTION RESEARCH GROUPS TOPICS & PERSONNEL KNOWLEDGE CREATION Prof. E. Bueno and P. Morcillo UAM

13. Scientific and Technological Production of the Community of Madrid Since 1994 updated every year Bibliometric indicators International and National Databases All disciplines Scientific publications and patents Institutional and research groups performance Sectoral analysis by science domain Measuring science technology production trends Regional comparative analysis-relative national and international position Prof. I. Gómez-Caridad et al CINDOC

14. PROGRAMME EVALUATION Sectoral studies of different PRICIT actions Comparative analysis against National R&D Plan Passed performance: regional, national and international Whole analysis under Foresight action: Three kind of discussion groups Wider participation through internet

15. TECHNOLOGICAL TRENDS ANALYSIS PRICIT evaluation Attractiveness and relative position S/T Capabilities Industrial & social demand Key Technologies- Science-Technology basis WARNINGS INDICATORS RECOMENDED ACTIONS Regional Programme for Research –IV PRICIT (2004-2007) FORESIGHT LINK Selection criteria

16. R&D Regional Programme - PRICITTechnology watch link :Innovation Circles The aim is to set up and develop technological watching and economic intelligence routines and to promote co-operation between companies and research groups in order to facilitate technology transfer. Circles already existing are: Agrofood, Biotechnology, Energy, Materials and Production Technologies, Environment, Microsystems and Nanotechnologies and Information Technologies

17. GRANT EVALUATION PROCESS “present” EXTERNAL EVALUATION ANEP R&D PROPOSAL SCIENTIFIC QUALITY FINAL EVALUATION REPORT EVALUATION SCIENTIFIC CO-ORDINATORS GENERAL DIRECTORATE FOR RESEARCH THEMATIC CLASSIFICATION RELEVANCE POTENCIAL IMPACT SCOPE OF THE CALL- INTERNAL EVALUATIÓN DGR -CM EVALUATION CRITERIA AND S&T PRIORITIES GENERAL DIRECTORATE FOR RESEARCH FINAL SELECTION

18. The attenuation effect of ex -ante evaluation on planning Combining excellence and innovation success Ad hoc criteria linked to strategic goals Gaining transparency maintainig confidentiality More diverse and open vision in the evaluation process How to cope with complementarity national, european one Funding science supply instead social demand Balance scientific production and technological transfer activities Need to continuous integrated monitoring against deliverables Formative evaluation, visible evaluation committees Integrated vision, interactive process Need to measure impacts of regional policy EVALUATION PROCESS “future needs and problems to solve”