Validation of Forecasts of Scientific Indicators: The Case of World Leadership of Science

1. Validation of Forecasts of Scientific Indicators: The Case of World Leadership of Science R. D. (Duane) Shelton ITRI & WTEC Baltimore, Maryland, USA Atlanta Conference on Science and Innovation Policy, Sep., 2013

2. Outline Introduction: Theories in Scientometrics Indicator Extrapolation Validations: When will China again lead the world? Validation and Refinement of the Shelton Model for Paper Shares Conclusion: Opportunities to Build More Models

3. Uncle Sam Worries About the Rise of China And so does John Bull, Deutcher Michel, Marianne, Bharat Mata, …

4. Purposes Provide indicators to evaluate current national leadership of S&T (2009 data mostly). An art. Validate earlier forecasts that were based on 2005 data. A bit of science, but mostly art. Focus on a theory to explain why some countries excel in scientific publication. A little more science. You can extend to other indicators: PhDs, patenting, hi-tech exports, … The next few slides review the forecasts in 2009.

5. Hi-Tech Export Share (Example of 2009 Forecast: China Would Lead by 2010) This was typical: China starting far below, but rising rapidly. It was easy to extrapolate underlying exports graphically, but shares have to add to 100%. (Shelton & Foland 2009)

6. Input Indicators (2005 data in 2009 paper) Average annual percentage rates of change in parentheses. Blue emphasizes very rapid growth. GERD = gross expenditures on R&D. (Shelton & Foland 2009)

7. Output Indicators (2005 Data in 2009 paper) Average annual percentage rates of change in parentheses. (Shelton & Foland, 2009)

8. Summary of National Positions In 2005 data the leader was: US: GERD, researchers, impacts, patents, hi-tech exports EU: papers in SCI, S&E PhDs, Nobels PRC: trade balance But extrapolation forecasts done in 2009 showed that the PRC would gain: Lead hi-tech exports by 2010 Lead in researchers by 2010 Pass US in S&E PhDs by 2015 (EU will still lead) Pass EU in GERD by 2015 (US will still lead) Lead in papers in SCI by 2017 Let’s update with new data.

9. Updating the Main Forecasts (Highlights First) • Hi-Tech market share forecast was correct • Researcher indicator not evaluated because of data problems • Patents still show China far behind, but … • PhD grads on track but crossover delayed by a year • GERD on track, but crossover advanced by a year • Papers on track: China should pass the US by 2017 (some say its growth has slowed)

10. It was Easy to Forecast that China Would Take the Lead In High Tech Exports The vertical line (at 2005) divides data used for 2009 forecasts, and actuals since then. You could build a model for this, but you might want to collaborate with me since I have one started.

11. PCT (International) Patents are a Good Indicator of Applied Research You could put this data into MiniTab and forecast the crossover. Or build a model based on a GERD or BERD driver to connect patents to investments. (Shelton & Monbo, 2012)

12. S&E PhD Grads (2009 Paper) EU = Germany and UK only; EU27 curve would be far above the US one.

13. Forecast Based on Latest Data: When Will the PRC Pass the US? The PRC values after 3008 are estimated as 64.5% of their totals. You could build a model connecting this to the driver: the number of researchers—a birth-death differential equation!

14. Focus on Model for Papers and its GERD Driver • This really comes from applied math modeling • Regression confirms connection • GERD is a good explanatory variable for papers. • Later I found that the government component of GERD, and the higher education spending component are slightly better. This might be a good time to explain WHY China is rising so fast in all these indicators.

15. Why China is Rising in Science: Money is the Engine It gets worse. The US sharply cut its real investment in R&D after 2008; the EU cut somewhat. This will speed their fall relative to the PRC. GERD = gross national expenditures on R&D (OECD 2013).

16. GERD Share With Forecasts (2009 paper) This is based on an extrapolation of GERD, then shares of all OECD countries are constrained to add to 100%.

17. Validation: My Short Term Forecasts of GERD Shares are Not Bad F = Forecasts in 2009 based on 2005 data; A = Actuals. The PRC and EU are a little more than forecast, but the US is less.

18. 2011 Data is Now Available; So When are These Likely to Cross? We can use MiniTab to forecast, but we need to decide if growth is linear or exponential.

19. It’s Hard to Tell if a Curve is Linear or Exponential at First, Especially if Data is Noisy Test: linear growth has a constant slope, geometric growth has an increasing slope. (Finite differences are the discrete derivatives.)

20. Finite differences Show China’s GERD growth is NOT Linear--Its Slope Increases In response to the Great Recession (2008) the West cut back R&D investments; the Chinese accelerated them. Geometric growth will be used for China in forecasts.

21. The PRC is Now Forecasted to Lead the World in R&D Investments by 2016 This is one year earlier than I forecasted in 2009. Linear vs. geometric doesn’t much matter for US or EU. These MiniTab forecasts are based on 1995-2011 data.

22. SCI Publications (2009 Paper) From NSF S&EI 2008, fractional count.

23. My Short Term Paper Share Forecasts are Not Bad Unlike the earlier forecasts, these are not just extrapolations of the output indicator. They are based on what I modestly call the Shelton Model. How does it work?

24. Model of a National Scientific Enterprise “The Black Box” S&T Outputs Resources In Indicators measure inputs and outputs Multiple linear regression can identify which inputs are most important This is a “scientometric model” similar to an econometric model.

25. More Detailed Model of Publication System (Inside the Black Box) $ Inputs Papers Published p1 g1 US EU Journal Editors PRC ROW G (total) P (total) mi = pi/P Paper share wi = gi/G GERD share National Research Systems -- Fairly Independent Highly Interdependent Paper Selection

26. A Simple Model for Country i mi = k iwi • mi is share of papers published (fractional basis) • wi is the share of GERD for the OECD Group • k i is a "constant" of proportionality; it differs by country. • k i is also the efficiency of country i in producing papers per $1 million in GERD, normalized by the OECD average efficiency. • For data in a single year the equation is an identity, but it is most useful over a range of years IF k i is approximately constant

27. Relative Efficiency for Fractional Papers in the OECDg: ki = Paper Share / GERD Share This continues to be flat, indicating that the Shelton Model still works for these countries. Chinese ki changed from the 2009 paper because its PPP weight changed. It is now as efficient as the US.

28. Use of Shelton Model • It connects a policy input (investments) to a science indicator output (papers) • It permits better forecasts since the investments can be more easily predicted. They are often set by published policy goals, and some countries like China actually meet their goals. • Thus I forecast GERD and use the model to forecast papers. • I don’t simply extrapolate papers, as I did with the other indicators.

29. Shelton Continues to Forecast China Passing US by 2017 and EU28 by 2018 to Lead in the SCI. Fractional count data from SEI2012 through 2009, forecasts after 2009 based on OECD (2013) GERD data through 2011. The US crossover will occur a year or two sooner if the US continues its folly of cutting real investment in R&D.

30. Further Work in Science and Innovation Policy • You can develop scientometric models for other indicators, similar to econometric models, to provide policy levers • Here are some hints: [the best explanatory variables] • Patents [Industrial GERD, BERD with lags] • PhD Graduates in S&T [Researchers] • International Market Share in Hi-Tech Products [GERD, BERD with lags] • Improved Models for Papers [Govt. GERD, HERD] • Impacts and Citations [?] • Be cautious: like economics, scientometrics is social science, not rocket science • PS: I’ll be glad to help you

31. Update of Forecasts of National Positions In 2009 data the leader was: US: GERD, impacts, patents EU: papers in SCI, S&E PhDs PRC: trade balance, hi-tech trade But current forecasts predict that the PRC will gain: Lead in GERD by 2016 Pass US in S&E PhDs by 2015 (EU will still lead) Lead in papers in SCI by 2018

32. Linear Forecasts Depend on What Interval You Use Based on 1995-2011 Based on 2008-2011

33. Conclusions Forecasts predict that China will soon pass the US and EU in key indicators I forecast that, if present trends continue, China will regain its historical leadership of world science and technology by 2017. It’s the Central Kingdom, after all. But there are some caveats: Qualitative assessments of Chinese science are not quite so positive, e.g. those of WTEC Black swan events also fill Chinese history: financial or political upheavals could derail their progress—neither would be good for the West, either.

34. Key References • Shelton, RD (2008)  Relations between national research investment input and publication output: Application to an American paradox.  Scientometrics  Vol. 74 No. 2, 191-205, Feb., 2008. • Shelton, RD & P Foland, (2009) The race for world leadership in science and technology: status and forecasts. Proceedings of the 12th International Conference on Scientometrics and Informetrics 369-380. Rio de Janerio, Also in Chinese in Science Focus (2010) 5:1 1-9. • Wilsdon, J, et al. (2011) Knowledge, networks, and nations: Global scientific collaboration in the 21st century. London: The Royal Society. • Shelton, RD & L. Leydesdorff, (2012) Publish or Patent: Bibliometric evidence for empirical trade-offs in national funding strategies. Journal of the American Society for Information Science and Technology. Vol. 63(3): 498-511. • Leydesdorff, L. (2012) World shares of publications of the USA, EU-27, and China compared and predicted using the new interface of the Web-of-Science versus Scopus. El Professional de la información 21 (1). • Shelton, RD & S. Monbo (2012). Input-output modelling and simulation of scientific indicators: A focus on patents, Proceedings of the 17thInternational Conference on Science and Technology Indicators, pp. 756-767. Montreal. • Zhou, P (2013) The growth momentum of China in producing international scientific publications seems to have slowed down. Information Processing and Management 49 (4) 1049 – 105. • Fu, J; R Frietsch, U Tagscherer (2013) : Publication activity in the Science Citation Index Expanded (SCIE) database in the context of Chinese science and technology policy from 1977 to 2012, Fraunhofer ISI Discussion Papers Innovation Systems and Policy Analysis, No. 35. • http://stats.oecd.org/Index.aspx?DataSetCode=MSTI_PUB MSTI 2013-1 database from the OECD Accessed 8/29/13

35. Appendix: Extra Slides More info at http://itri2.org/Atlanta2/

36. Update of Top National Positions in the SCIE Some say the Chinese increase has changed from exponential to linear (or even ended!), but it is hard to tell here. (Whole WoS counts, and share of the world, not the fractional shares of OECDg in the Shelton Model.)

37. Linear Extrapolation Details Some indicators are zero sum--GERD share, for example. Results need to be constrained to add to 100%.

38. Foreign Industry R&D Funding Model for Applications and Grants from R&D Inputs Multiplier Delay Patent Office Applications Grants Backlog Queue Server Abandonments Delay GPSS/H is used for simulation Multiplier U.S. Industry R&D Funding Shelton & Monbo (2012)

39. PRC Papers in the WoS: Whole Counts This also shows that Chinese paper growth is not linear, but continues to grow exponentially

40. Total sales is not a zero-sum game—the hi-tech sector is growing rapidly worldwide

41. These discrete derivatives show roughly linear growth, but all rates slowed after mid-decade, except for the US 2011 point.

42. This discrete derivative shows whether changes are arithmetic (linear) or geometric (perhaps exponential) Since its slope increases, Chinese GERD continues to grow geometrically, while the average in the West is only linear.

43. This shows that the rate of new grads is approximately proportional to the total number of the current stock of researchers. This is a birth-death process, leading to a simple linear first-order differential equation model.

44. This birth-death model works for some of Europe’s largest countries, but not all

45. Even in Percent Shares, the PRC is still rising faster than linear, while the others decrease fairly linearly, not keeping up with the rise in the overall SCIE In the limit when the shares are US = EU27 = 0%, and the PRC = 100%, all these first differences must flatten out!

46. Scientific PublicationsINSPEC (Physical Sciences) As forecast, China took the lead in 2008. It now has double the papers as the US.

47. Triadic Patents Continue to Show China Rising Rapidly From a Small Base

48. This MiniTab plot also shows Chinese GERD increases faster than linear

49. This is a new Ki-prime based on the WoS Whole Counts, 3 types of pubs, share of whole world, but divided by GERD share of OECDg. I need to find out why the PRC efficiency changes here, perhaps because of their jump in investment in 2009

50. A linear increase must result in constant finite differences, but the Chinese pattern shows a continuing increase in this discrete version of the derivative