Mapping of S&T excellence in Europe Brussels, 23 November 2000

1. Mapping of S&T excellence in EuropeBrussels, 23 November 2000 Mapping of S&T excellence in Europe: building on the Portuguese experience in evaluating research excellence Pedro Conceicão and Manuel Heitor CENTER FOR INNOVATION, TECHNOLOGY AND POLICY RESEARCH, IN+ Instituto Superior Técnico, Technical University of Lisbon http://in3.dem.ist.utl.pt

2. The scope • acceleratingtechnological change: knowledge creation and diffusion is increasingly recognized as a major driver of economic growth • strategic development of technology platforms: The need to access to new knowledge and the science base, together with the requirements to speed-up knowledge diffusion and technology commercialization • the context:collaborative learning and the emergence of the learning society! The result: the need for a competitive science base

3. The question What can we learn from the portuguese experience in research evaluation, in a way to stimulate theEuropean Research Areaand to promote research excellence in Europe?

4. Knowledge InstitutionsLearning Organisations Intellectual PropertyLearning Networks The locus of excellence? Varied and multidimensional information! The need to identify the potential for excellence in the coming decades: international peer review Our argument... • The scope:...the globalized “learning society”! BUT, stimulating the inclusive development ! Conceicão, Gibson, Heitor & Sirilli (2000)

5. The contents 1. The objective 2. A Portrait in Figures: A Quantitative Perspective of Portuguese S&T in OECD 3. The evaluation of research excellence in Portugal 4. The context: Building the European Research Area 5. Concluding

6. Resources: Scale vs Intensity R&D Expenditure (OECD) 0,04 Sweden 0,035 0,03 Japan Finland US France 0,025 the Netherlands Germany Denmark Intensity- Share of GDP spent on R&D 0,02 UK Belgium Ireland 0,015 Austria Italy 0,01 Portugal Spain 0,005 Greece 0 100 1000 10000 100000 1000000 Scale- Total Expenditure in R&D ($PPP; logarithmic scale)

7. Output: publications vs expenditure

8. Scientific and technological capacities in OECD Fonte: Caracostas & Muldur (1998)

9. Innovation in EU source: Community Innovation Survey, OCT-PT

10. Growth of Knowledge-based industries 14 Korea 12 10 8 Average Annual Real Value Added Growth of Knowledge Based Industries (1985-share year) Portugal** 6 UK* Denmark 4 Japan Mexico Austria Germany Canada NL* Greece* Spain*** US Italy France Belgium Sweden*** 2 Norway Denmark 0 0 10 20 30 40 50 60 Share in Business Sector Value Added of Knowledge Based Industries (share year 1996 except: *1995;**1993; ***1994) OECD(2000)

11. Portugal Japan Netherlands Spain Ireland Austria Mexico Norway Italy Sweden Finland Germany Canada UK Iceland Belgium France Switzerland Denmark Korea US Australia -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 Change in the Amount of Tax Subsidies for 1 US Dollar of R&D Incentives to BERD1990-1998 SOURCE: OCDE (1999).

12. …further evidence Le Monde, 21 November 2000 Based on positive correlation ofproductivity growthand theincrease in BERD: “les bons élèves da le nouvelle économie”: ...en Europe, la Scandinavie, l’Irlande et le Portugal tirent le mieux leur épingle du jeu, ...

13. The issue…Evaluating scientific excellence in Portugal We argue:Portugal is facing the growing importance of digital technologies The evidence:Structural debilities in the education and R&D system still persist, despite the recent growth of R&D expenditures and the effort to provide incentives for private spending on R&D The challenges:- development of new skills - involvement in international networks of excellence The question for S&T:the extent to which the existing research excellence in Portugal allows the involvement in European (and international) networks of excellence?

14. The contents 1. The objective 2. A Portrait in Figures: A Quantitative Perspective of Portuguese S&T in OECD 3. The evaluation of research excellence in Portugal 4. The context: Building the European Research Area 5. Concluding

15. Evaluating scientific excellence in Portugal:the concept • critical analysis of research activities, based on recommendations from external experts with experience in scientific assessment. • a process of regular assessment, encouraging the development of a dynamic scientific and technological base: • launching the process:1996 • promoting reorganization of the S&T system:1997/98 • a continued process :1999

16. Evaluating scientific excellence in Portugal:the methodology • International peer-review: • panels of foreign scientists using international reference criteria Evaluation based on: documentation Results quality and quantity relative to number of PhD’s Relevance of the research activity Internationalization - quality and quantity visit Potential for developing excellence Intrinsic merit of the Unit activities Attitudes and work environment young researchers Resources for the research activity Multidisciplinarity:multiple peer review

17. Evaluating scientific excellence in Portugal:the methodology Grades Description Research activities at a high international level, Excellent with publications in internationally leading journals Research activities at a good international level and at a high national level, Very Good with publications in inte internationally leading journals Research activities at a high national level and at a fair international level, Good with publications in internationally well - known journals Research activities at a fair national level, Fair with publications only partially - known journals in internationally well Poor Research activities of insufficient quality

18. Evaluating scientific excellence in Portugal:areas and number of PhD’s

19. Evaluating scientific excellence in Portugalquantitative results:grading

20. Evaluating scientific excellence in Portugal:PhD’s per grade 1800 1565 1600 1400 1273 1200 988 1000 888 1996 862 1999 800 730 600 474 400 329 200 87 0 0 Excellent Very Good Good Fair Poor

21. Evaluating scientific excellence in Portugal:the issue…2 the usage of indicators of shares of national resources used in R&D (as inputs) and the usage of bibliometric indicators and number of patents (as outputs) are likely to consistently bias an accurate portrait of the size and, especially, quality of Portuguese research The evidence:a number of excellent R&D units The method:international peer-review

22. The contents 1. The objective 2. A Portrait in Figures: A Quantitative Perspective of Portuguese S&T in OECD 3. The evaluation of research excellence in Portugal 4. The context: Building the European Research Area 5. Concluding

23. Discussing S&T excellence in Europe… 3 main ideas: European Research Areaas a way to promote the inclusive development of the European science base • Scale and diversity • Competitiveness at average level Benchmarking:learning by comparing • A collective process leading to mutual learning • Metrics for knowledge Centres of Excellence with a European dimension • Networks of excellence

24. Conclusions Mapping excellence requires the identification of the locus of excellence across Europe. The information needed to identify these poles of excellence is, naturally,variedandmultidimensional • We argue that: • in the case of Portugal, scientific and technological excellence may be lost in aggregate quantitative indicators traditionally used to measure scientific and technological activities. • peer review, with international criteria, is able to identify to enablers of new competences and the potential to develop excellence in the coming decades. • European Research Area requires poles of scientific competences, and the development of networks of research excellence

25. Mapping of S&T excellence in EuropeBrussels, 23 November 2000 ...debate

26. Human Developmentand Inputs to S&T

27. Public vs private R&D expenditures 0,1 France Netherlands 0,075 Germany Denmark Finland Poland Norway US Japan Public R&D Expenditures per capita Korea Canada UK 0,05 France UK US Netherlands Canada Germany Czech Rep Finland Denmark Ireland 0,025 Japan Norway Sweden Spain Portugal Ireland Sweden Belgium New Zealand Hungary Spain Portugal Mexico Turkey 0 0 0,125 0,25 0,375 0,5 Private R&D Expenditures per capita FR D JP USA UK P97 P95 ES SE IR P81

28. Regional diversity in Western Europe Source:Sixth Periodic Report DG XVI, 1998