Conduct of an R&D Survey: Survey procedures for each sector

Conduct of an R&D Survey: Survey procedures for each sector RICYT-UIS-CONACYT Training Workshop on STI Indicators San Salvador, El Salvador 4-5 November 2015 Rohan Pathirage, UIS

Outline • Scope of R&D surveys • Government sector • Higher education sector • Business enterprise sector • Private non-profit sector

Survey procedures for the sectors • Statistics on R&D require regular, systematic and harmonised special surveys. • Other sources provide information, but: • concepts of R&D used often different from FM concepts • concepts may change over time • very difficult to obtain all data for the same period • difficult to avoid double counting when tracking flows from financial statements and other sources • Estimates are a necessary supplement to surveys • especially in higher education sector

Scope of R&D surveys • R&D surveys should identify and measure all financial and personnel resources devoted to all R&D activities in all R&D units, at all levels. • R&D surveys are mainly addressed to R&D-performing units. • Frascati Manual (chapter 7) and this Guide (chapter 4) only address performer-based surveys.

Scope of R&D surveys cont… • Exhaustive survey not possible in most countries. • Constraints include: • number of respondents may have to be restricted to keep costs down. • R&D survey may have to be taken in conjunction with another survey. • surveys of some groups may require the participation of other agencies with different data needs and hence different questions for respondents.

Scope of R&D surveys cont… • Size and structure of R&D capacities vary. Every country has different constraints. • One size does not fit all: advice is therefore of general nature. • Each sector has different management styles, approaches and institutional culture. • Consider the existing norms in relation to data exchange.

Scope of R&D surveys cont… • Which sectors are to be covered?: public and/or private? Business, Government, Higher education, Private non profit? • Method?: census, sample or purposive?; sector specific questionnaires? • Who is the target of the survey? How to develop the survey registers? • Who will complete the survey questionnaire? • How contact with organizations is done? • How the survey will be conducted?: interviews, postal, email correspondence, web based, etc.? • What is the substantive set of data that will be requested? • Need to consider the sector and the size and complexity of the organizations.

Government sector • Units to include in surveys are: • Public R&D institutes: Public research institutes (PRIs); Department-based research institutions (DBRIs). • R&D activities of general administrations of central or state government. • Public institutions dealing with STS: statistical, meteorological, geological and other public services, museums, hospitals. • R&D activities at the municipality level.

Government sector cont… • Conduct a census of relevant organizations/units known or assumed to perform R&D. • Identification generally easy. Sources for survey registers: • Registers of government departments, research institutions, statutory bodies, list of S&T services institutions. • Associations (trade, academic); Learned societies. • Registers or databases of scientists and engineers. • Database of research grants; Databases of scientific publications. • Science and technology management information system (STMIS). • Patents and other IP documents.

Government sector cont… • Department-based research institutions (DBRIs) • Issues: non-availability of information in compiled form • Contact point: Senior civil servants: Director-General or Permanent Secretary • Public research institutes (PRIs) • Contact point: Chief Executive Officer or executive responsible for research management • It is common that PRIs/DBRIs report that all their staff involved in research and that their FTE is equal to, or close to 100%. However many PRIs/DBRIs are engaged in R&D, scientific and technological services (STS) and consulting activities. The STS activities may or may not be part of R&D projects, and where they are not, this quantum of activity should be excluded. • Public organizations primarily engaged in scientific and technological services (STS) often undertake some research in connection with this activity. Such research activities should be clearly identified and be captured in R&D survey.

Higher education sector • Units to cover: All universities and corresponding institutions, especially those awarding degrees at the doctorate level. Also other institutions in the sector known or assumed to perform R&D. • Could use smaller units, such as departments or institutes of the university, as statistical units. • Conduct a census of relevant organizations/units known or assumed to perform R&D. • Identification generally easy. Sources for survey registers: • List of higher education institutions • Same sources as used in Government sector. • Researcher CV database; Publications databases (Web of Science™ or Scopus™).

Higher education sectorcont… • Different degree of autonomy • Staff employed as civil servants – list of employees is available • Academics directly employed by HEI – staff details are protected • Contact point: through Vice Chancellor/President or Deputy Vice Chancellor/Pro-Vice-Chancellor for Research, Dean of Faculties, Dean of Research or, Head of Departments, etc.

Higher education sector cont… • Higher Education institutes (HEIs) are the main seat of R&D activity. • Determining the headcount of staff that are active researchers, and estimating their time spent on research and also obtaining a good estimate of the overall research expenditure is key (refer to annex 2 of FM for more details on time-use surveys and other methods of estimating shares of R&D - R&D coefficients- in total activities in the higher education sector).

Business enterprise sector • The enterprise is recommended as the main statistical unit in the business enterprise sector. • Some enterprises perform R&D on a regular basis from year to year, and may have one or several R&D units • Other enterprises perform R&D only occasionally • Units to cover: All enterprises performing R&D, either continuously or occasionally, should be included in R&D surveys.

Business enterprise sector: first approach • A census-based survey of large enterprises and a sample of smaller ones in order to identify R&D performers and request the information from them. • Should be based on business register. • Sample could belong to a certain population (in terms of industry and size class). • R&D performed in the past in the enterprise is not considered. • This is the approach followed in innovation surveys. • Very small enterprises and enterprises in certain less R&D-intensive industries often excluded for cost reasons. • When the sample size is very small, estimates may be less reliable, owing to raising factors. • Method not strictly followed in any country.

Business enterprise sector: second approach • If no comprehensive and up-to-date business register; no directory of companies that perform R&D  census estimate through a random sample is not possible. • Conduct a purposive survey that deliberately sets out to identify R&D performers (i.e. cover all firms known or supposed to perform R&D, based on a business register of R&D performing enterprises). • How to detect R&D activity in Business? Sources for Business register of firms that perform R&D.

Business enterprise sector:second approach cont… • Sources for Business register of R&D performing enterprises: • Registers of publicly funded research grants / Directories of R&D laboratories • Lists of enterprises reporting R&D activities in previous R&D surveys, or in innovation surveys / enterprise surveys. • Industrial research associations / Professional associations / Chamber of Commerce / Trade associations • National stock exchange • Company annual reports / Trade journals • Register of approved clinical trials / GMO trials • Lists of enterprises claiming tax deductions for R&D

Business enterprise sector: joint approach • To include all firms known or supposed to perform R&D, based on a business register of R&D performing enterprises: Purposive survey. • To identify R&D performers not known or supposed to perform R&D by a census/sample of all other firms: • In selected industries (given in the next slide). • In principle, enterprises in all size classes should be included, but if a cut-off point is necessary, it should be at ten employees. • Contact point: through CEO, or a divisional head.

Business enterprise sector: joint approach cont… Industries to be included: Plus any other industries relevant for the country: e.g. Agriculture (ISIC Rev. 3, 01, 02, 05), etc.

Business enterprise sector cont… • Some typical issues: • R&D performed in business sector remains low in many developing and emerging economies; Even in developed countries, 5-10% of firms carry out 90% of R&D. • Although business sector is central to the goal of innovation, studies have shown that only a few percentage of firms are engaged in regular R&D activities. Many firms that innovate do not carry out in-house R&D. • Necessary to understand the structure of companies to avoid double counting (holding companies or diversified industrial groups). • Necessary to understand the structure of companies to avoid double counting (holding companies or diversified industrial groups). • Developing the business register of firms that perform R&D: start with large firms in sub-sectors that ‘usually’ involve some R&D: petrochemicals, chemicals, mining and mineral processing, pharmaceuticals, electrical, electronics and software. Ask ‘who in your sub-sector is also doing R&D, and how do we contact them?’

Business enterprise sector cont… • Some typical issues: cont … • Exclude holding companies, construction, retail, and utilities as sub-sectors likely to perform little or no R&D. • Need direct interaction with firms to verify the existence of R&D activities. • Large firms/MNC - discussion with the Chief Financial Officer or Chief Technology Officer; Missing a large firm might result in significant error. • Publicly-owned businesses play a major role in R&D in some developing countries: • should consider issuing data for ‘publicly-owned businesses’ separately from the ‘fully private enterprise sector’. • private enterprises could also be disaggregated by ownership, in particular the various degrees of foreign ownership. • Service sector often under-reported: try to engage with leading banks, insurers, mobile telephony and ICT companies.

Private non-profit sector • Differs significantly country by country; same challenges as in business. • Difficulty in identifying PNPs engaged in R&D; Not clear about, status, ownership. • Engaged in wide range of activities; Perform in-house R&D as well as contract R&D. • Sources for identifying possible survey respondents: mainly the same as for the government sector. • Register information may be less comprehensive and could be completed by information from researchers or research administrations about the PNP institutions known to perform or support R&D. • Conduct a purposive survey.

R&D Manager Better understanding of R&D and FM norms But may not be able to supply exact figures Accountant or personnel manager May not refer exactly to R&D as defined in FM But able to supply exact figures Who is the right respondent? • Cooperation of all three may be needed • Useful to identify in advance the person responsible for providing information and for co-ordinating information from smaller sub-units.

Who is the right respondent? cont… • Government: PRIs, DBRIs: In both cases, questionnaire should be completed by the Head of research or research manager(s) incorporation with Accountant/personnel manger. • Higher education: Questionnaire should be completed by Dean of Faculties or Dean/Head of research/departments in collaboration with Account/personnel departments. • Business enterprise: Questionnaire might be completed by the CEO, or a technology/production manager. It is rare for staff in human resources or finance to have such knowledge and information.

Thank you! http://www.uis.unesco.org r.pathirage@unesco.org

Conduct of an R&D Survey: Survey procedures for each sector