The fork in the road for innovation measurement: which way should we go?

1. The fork in the road for innovation measurement: which way should we go? Jonathan Haskel Imperial College Business School, Imperial College London and CEPR, UKIRC j.haskel@ic.ac.uk National Academy of Sciences, Washington DC, 11th July 2011 Slides plus spares, links to papers etc. in spares section

2. The exam questions(to be answered in final slide…) • The objective of this session is to identify • recent developments in measuring STI and • what is currently planned for the future.  • Discussion should reveal • what has been successfully and unsuccessfully measured.  • What are critical bottlenecks and perceived opportunities? • What global STI metrics and indicators should NCSES develop in the near and medium term (the next 5-10 years)?

3. The fork in the road for innovation measurement • Approach 1: EU-type innovation surveys • Tried for 15+ years in EU, Applied in latest US work • Key questions • “have you process/product innovated?” • Spending on innovation e.g. software, marketing, training • Potpourri of other questions • Information for innovation (e.g. importance of information from universities high/med/low) • What stopped you from innovating? • Approach 2: extended R&D/spending type surveys • Follows Griliches/Jorgenson/CHS/Nat Accounts to measure knowledge capital • Implementation: software and (forthcoming) R&D spend • CHS agenda: need coinvestments with R&D e.g. design, training, marketing, business process • Current method: potpourri of surveys • UK: intangibles investment survey, similar style to R&D survey • Kauffman

4. What theory do we have to guide measurement? • Innovation as TFP i.e. output net of tangible K and labour • Logically consistent, suggests measurement (e.g. depreciation, investment) • Appropriate if all knowledge from free spillovers • Corrado, Hulten, Sichel • Recognises innovation is routinised in many firms e.g. organised spending on marketing, training , design etc. • Helps understand innovation in non-R&D industries e.g. finance • Strength: CHS categories are matched to functional bodies in (large) firms e.g. HR dept for training , marketing dept for brand etc. • Still scope for knowledge spillovers • Measurement implications • Spending, investment, depreciation, asset and rental prices

5. European Innovation survey questionnaires • UK example: product innovation • Processinnovation

6. What do innovation surveys find? Michael Mandel: “You can’t be an innovative economy if only 9% of your companies are innovating” US data: Boroush (2010), Other: OECD Measuring innovation (2010), p.26

7. What does the “have you innovated” question measure? • Other findings • the total is procyclical: rises strongly in IT boom end of 1990s • Item non-response can be very high e.g. 70% to intangible spending questions • Large firms show poor response rates • So what does the question measure? Crespi , Criscuolo, Haskel (2007, Appendix 1), UK CIS3 (98-00) • 45% of firms reported technological product/process • they were asked for text describing their innovation • They are given instructions on what to include and exclude e.g. include “robotised welding”, exclude “packaging” • Analysis of 874 firms showed

8. The “have you process innovated “ question measures new capital equipment • Summary: • of firms who provided the information, reported process innovation is mostly new capital spending • This is why measured “innovation” is highest in IT boom • Conjecture: we would get same/better(?) information from a standard investment survey (+ good deflators)

9. An alternative approach guided by CHS • What does this framework suggest you need to know? • Knowledge services within the firm = knowledge capital stock and rental rates • Innovation spending on broad range of knowledge assets: software, design etc. • Convert to investment • Convert to real stock: need depreciation and deflators • Implication: need a spending survey. Can piggy back onto R&D survey or new one (or modify innovation survey) • Knowledge from outside the firm • Information flows • Patents is one source: but service sector? • CIS information flows

10. Possible model: Investment in Intangible Assets Survey(www.ceriba.org.uk/bin/view/CERIBA/InvestIntangAssetsSurvey)) • Extension of UK R&D survey, • Autumn 2009, through ONS: so linked to national accounts data and R&D and CIS survey • Survey follows (for large firms) typical business functions

11. Each section has a filter question which defines the asset with examples

12. Then asks purchased and own-account

13. Finally life lengths

14. Life lengths • All asset life lengths are greater than one year and range between 2.75 years for training and reputation to around 4.5 years for R&D.

15. My answers to the exam questions • The objective of this session is to identify • recent developments in measuring STI • US innovation survey raises puzzles for survey design. Intangible assets framework gaining policy traction. Some surveys moving in this direction. • what is currently planned for the future • Review by OECD of innovation surveys. More UK intangible surveys. Kauffman survey to continue? • Discussion should reveal • what has been successfully and unsuccessfully measured.  • yes/no innovation questions have not worked. I • Intangible spending can work if asked to business functions, own-account explained. • Business process has not worked in UK survey • What are critical bottlenecks and perceived opportunities? • Shortages of funding for more surveys. Opportunity (a) in EU, convert Innovation surveys to intangible spending surveys, (b) in US to convert supplement to BERDIS to intangible spending survey. • General lesson: take best practice from each survey • What global STI metrics and indicators should NCSES develop in the near and medium term (the next 5-10 years)? • Intangible spending, life lengths. Integrate with National Accounts work on deflators, service sector output.

16. Spares

17. What do innovation surveys find? US data: Boroush (2010), Other: OECD Measuring innovation (2010), p.26

18. Crespi et al, 2008. What do firms write down as their most significant process innovation? • Replies • 53% are other, mostly including capital equipment (e.g. steel wire machine, spectrophotometer..) • 23% are hardware related (computer, CAD, PC) and software related (website, email, software, internet etc.) • Summary: of firms who provided the information, reported process innovation it new capital spending • Not surprising that measured “innovation” is highest in IT boom • We likely get equivalent information from a standard investment survey (and spend resources on better deflators)

19. The future • Take best practice from elements of different surveys • Innovation surveys • Don’t ask % new, % innovation • Do ask information sources • R&D/IAS • Do ask for spending • Keep structure of questionaire allied with business function: ICT dept for software, HR dept for training, Marketing dept for branding • Complement with • What do firms do? Innovation data at project level ? • What do young firms do? Likely very intangible intensive. Kauffman survey. • What do consultants do, especially intangible consultants? • What do financial institutions do esp young firms, they are trying to value innovation? • Measuring business process huge challenge • Chance for NAS/OECD to lead as OECD did on software