Evaluation of the National Natural Science Foundation of China

1. Evaluation of the National Natural Science Foundation of China Erik Arnold American Evaluation Association Anaheim, CA 3 November 2011

2. Some milestones • Cultural Revolution: S&T system effectively destroyed • 1978 National Science Conference: “modernisation of S&T is the key of the four modernisations”; legitimacy but little money • 1985-92 reforming the science system • CAS agency 1982-6; NSFC; marketisation of technology and business; decentralisation of strategy to research performers; reform of HR system to introduce merit-based pay • 1986, NSFC established, based on DFG/NSF model • 1992-8 integrating S&T to the economy, developing industry, universities and institutes • 1998-2005 building a national innovation system • Especially splitting off the industrially-focused parts of CAS, supporting SMEs, increased attention to human resource development and basic research

3. Medium- and Long Term Plan for National Science and Technology Development, 2006-2020 • Goal: to make China an innovation-driven economy by 2020 • High-priority clusters • Technologies for water, energy and environmental protection • IT, advanced materials and manufacturing • Biotechnologies and their applications • Space and marine technology • Basic sciences and frontier technology - Raise basic research to 15% of GERD by 2020 • 16 mission-driven megaprojects

4. Eight thrusts • A boost for investment in R&D • Tax incentives for investment in STI • Government procurement policy to promote innovation • Innovation based on assimilating imported advanced technology • Capacity-building in generating and protecting IPRs, standards • Building national infrastructure and platforms for STI • Cultivate and utilise talents for STI • Support endogenous innovation via financial measures

5. Key role of NSFC in basic research – the only bottom-up funder

6. NSFC main tasks • The combination of curiosity- and demand- driven research as a ‘dual driving force’. The first NSFC General Assembly stressed that its basic research funding was aimed to support economic development • Promoting the balanced, coordinated and sustainable development of academic disciplines in China • Emphasis on fostering talents • Facilitating international exchange and cooperation in basic research

7. Evaluation objectives • Provide an independent assessment of the overall performance of NSFC’s funding and management during the past 25 years, with a truly global perspective • Present key findings, lessons learned and recommendations to improve the NSFC’s funding and management performance as well as to achieve excellence in management • Develop a set of forward-looking guiding ideas, based on an international perspective, supporting NSFC’s strategic role within the NIS of China

8. Method • Review by an International Evaluation Committee • Academic chair • Thirteen researchers, including one evaluation professional • Interviews with NSFC and stakeholders • Extensive background report prepared by the National Centre for Science and Technology Evaluation (NCSTE) • Bibliometrics • Surveys • Interviews/focus groups • Document review

9. International Evaluation Committee • Prof Richard N ZARE (Chair), Stanford University, Chemistry • Prof HAN Qide (Vice Chair), Vice Chairman, National People’s Congress, Medicine • Prof Ernst-Ludwig WINNACKER (Vice Chair), Human Frontier Science Programme , Biochemistry • Prof Erik ARNOLD (Rapporteur), Technopolis Group; University of Twente, Research and Innovation Policy • Prof LU Yonglong (Rapporteur), Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, Environmental Science and Management • Prof XUE Lan (Rapporteur), Tsinghua University, Science and Technology Policy and Management • Prof Akito ARIMA, Chairperson, Japan Science Foundation, Nuclear Physics • Dr Richard A ANTHES, University Corporation for Atmospheric Research, Atmospheric Science • Prof Anthony K CHEETHAM, University of Cambridge, Materials Science • Prof MA Zhiming, Academy of Mathematics and Systems Science, Chinese Academy of Science, Mathematics • Prof Andrew (FA) SMITH, University of Adelaide, Agriculture, Food and Wine • Prof Jeannette M WING, Carnegie Mellon University, Computer and Information Sciences • Prof XU Zhihong, Peking University, Life Sciences

10. Stupendous growth in GERD (RMB billions)

11. BERD = 73% of GERD, but there is a low proportion of basic research in GERD

12. Building infrastructure, low labour cost component

13. Shifting the system towards research universities

14. A key innovation of NSFC was German-style peer review

15. Growth in projects and institutions funded across the research system

16. NSFC moved from general to more focused instruments

17. Age distribution of Principal Investigators

18. Discipline development

19. Women and ethnic minorities

20. Staff workload

21. Administrative efficiency is very high

22. Publications in the WoS

23. Relative impacts of publications relative to the World

24. Copublications Source: Thomson-Reuters

25. Growth, copublications 1999/2003 to 2004/9

26. UK/China co-publications grow faster than publications

27. From the UK perspective, there’s a quality penalty to pay for cooperation(mean impact factors 2000-2005) Source: RCUK

28. Surprisingly insular travel pattern

29. What the IEC said • Increase the share of basic research in GERD • NSFC needs more staff and resources • Bigger grants, more calls for proposals • Strengthen panels: interdisciplinarity; involve more foreigners • Ensure assessment is, and looks, confidential and ‘squeaky clean’ • More international connections; International Advisory Board • More flexible use of funding • More high-risk research

30. What the IEC didn’t talk about • Outcomes and impacts of NSFC funding (!) • Discipline development • The rich tradition of consultation in programming • Linking bottom-up and top-down funding approaches • Programme 1 versus Programme 2, in Swedish terminology • The systemic role of NSFC in developing the research and innovation community • NSFC’s role in wider policy development and implementation

31. Issues in the Chinese NIS • Learning to use all the investments • Raising quality while growing the system • Doing novel research in a top-down culture • Squeezing out ‘influence’ – is this still an issue? • Increasing basic research • Science-industry links, absorptive capacity • Autarchy vs globalisation and learning from abroad

32. Thank you http://www.technopolis-group.com/resources/downloads/reports/nsfc_evaluation_report.pdf technopolis |group| has offices in Amsterdam, Ankara, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn and Vienna