1 / 20

Innovation Surveys: Advice from the Oslo Manual

Innovation Surveys: Advice from the Oslo Manual. South Asian Regional Workshop on Science, Technology and Innovation Statistics Kathmandu, Nepal 6-9 December 2010. Ch 8 OM - Innovation Survey Procedures. Guidelines - collection and analysis of innovation data ; Comparable results ;

Download Presentation

Innovation Surveys: Advice from the Oslo Manual

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Innovation Surveys:Advice from the Oslo Manual South Asian Regional Workshop on Science, Technology and Innovation Statistics Kathmandu, Nepal6-9 December 2010

  2. Ch 8 OM - Innovation Survey Procedures • Guidelines - collection and analysis of innovation data; • Comparable results; • Particular circumstances may require other methodology comparability!

  3. Populations • The target population: • Business enterprise sector (goods-producing and services industries); • At a minimum, all statistical units with at least ten employees.

  4. Populations • The frame population: • Units from which a survey sample or census is drawn = frame population; • Basis: last year of the observation period; • Ideal frame = up-to-date official business register established for statistical purposes  NSOs; • If the register forms the basis for several surveys (innovation, R&D, business), the information collected can be restricted to issues specific to innovation.

  5. Survey methods • Mandatory surveys increase response rates; • Census or sample survey? • Sample surveys should be representative of the basic characteristics of the target population (industry, size, region) a stratified sample is necessary; • Census, though costly, might be unavoidable in some cases (legal requirement, small frame population, inclusion of all units in the frame with a certain number of employees).

  6. Survey methods • Domains (sub-populations) are subsets of the sampling strata; • Potential sub-populations: industry groupings, size classes, regions, units that engage in R&D and innovation-active; • Guidelines: • Same statistical units and classifications; • Consistent methods; • Documentation of deviations in data treatment or differences in the quality of the results (from the domains).

  7. Survey methods • Sampling techniques: • Stratified sample surveys (reliable results): based on the size and principal activity of the units; • Sampling fractions should not be the same for all strata: the sampling fraction of a stratum should be higher for more heterogeneous strata and for smallerstrata. • Cross-sections: standard approach - new random sample drawn from a given population; • Alternative/supplementary approach: panel data.

  8. Survey methods • Suitable respondents: • Various methods: postal surveys, electronic surveys,personalinterviews; • Questions are very specialised and can be answered by only a few people in the unit; • It is highly recommended to make a special effort to identify respondents by name before data collection starts.

  9. Survey methods • The questionnaire: • Pre-test before fieldwork; • Simple and short; • Order of the questions; • Questions on qualitativeindicators - binary or ordinal scale; • International innovation surveys: translation and design of the questionnaire; • Short-form questionnaires for units with little/no innovation activity previously reported.

  10. Survey methods • Combination of Innovation and R&D surveys: • Reduction in the overall response burden; • Scope for analysing the relations between R&D and innovation activities at the unit level; • Efficient method of increasing the frequency of innovation surveys; • It is possible to obtain reliable results for R&D expenditures; • Longer questionnaire; • Units not familiar with the concepts of R&D and innovation may confuse them; • The frames for the two surveys are generally different.

  11. Survey methods • Guidelines for conducting combined surveys: • The questionnaire should have two distinct sections; • Individual sections for R&D and innovation should be smaller than in separate surveys; • Comparisons of results from combined surveys with those from separate innovation surveys should be done with care; • Surveying methodsshould be reported; • Samples to carry out such surveys should be extracted from a common business register.

  12. Estimation of results • Weighting methods: • The simplest one is weighting by the inverse of the sampling fractions of the sampling units, corrected by the unit non-response; • Stratified sampling technique with different sampling fractionsweights should be calculated individually; • Commonly based on the number of enterprises in a stratum; • In international and other comparisons, be sure that the same weighting method is used.

  13. Estimation of results • Non-response: • Unit non-response: a reporting unit does not reply at all; • Item non-response: response rate to a specific question / % of blank or missing answers among the reporting units; • Disregarding missing values and applying simple weighting procedures based only on the responses received implicitly assumes that non-respondents are distributed in the same way as respondentsbiased results; • Possibility: imputation methods to estimate missing values on the basis of additional information.

  14. Presentation of results • Inferential analysis: • Drawing of conclusions about the target population; • The results should give a representative estimation of the situation; • Weighted results; • Unit non-response rate is very important. • Descriptive analysis: • Description of the statistical units in terms of their innovative or non-innovative activities without drawing any conclusions about theunderlying survey or target population; • No generalisation of the results; • Unit non-response rate is of minorimportance.

  15. Presentation of results • Errors: • Random errors due to the random process used to select the units; • Systematic errors containing all non-random errors (bias); • Results’ variance: it is recommended to calculate both (average) values for innovation indicators and also their coefficients of variation and/or confidence intervals; • Results presentation:metadata (including information on data collection procedure),samplingmethods,proceduresfordealingwithnon-responseandqualityindicators.

  16. Frequency of data collection • Innovation surveys: every two years; • When not economically feasible  three or four years; • Surveys must always specify an observation period for questions on innovation; • The length of the observation period for innovation surveys should not exceed three years nor be less than one year.

  17. Annex A - 5. Methodological issues for developing country contexts • Information system specificities: • Relative weakness of statistical systems • Absence of linkages between surveys and data sets; lack of official business registers information from other surveys cannot be used; • Involvement of NSOs; • When lacking, basic variables about firms’ performance can be included in the innovation survey - to enable further analysis.

  18. Annex A - 5. Methodological issues for developing country contexts • General methodological considerations: • Questionnaire design: • Separated sections - different respondents; • Guidance / definitions; • Language and the translation of technical terms; • Survey application: • In-person; • Trained personnel.

  19. Annex A - 5. Methodological issues for developing country contexts • General methodological considerations: • Frequency: • Every three to four years (e.g., timed to CISrounds); • Try to update a minimum set of variables every year; • The purpose of surveys needs to be clearly stated and the questions clearly formulated; • An adequate legislative base for the collection of innovation statistics can help ensure the success of such an exercise; • The results should be published and distributed widely.

  20. Thank you! http://www.uis.unesco.org l.marins@uis.unesco.org (CIS:http://www.oecd.org/dataoecd/37/39/37489901.pdf)

More Related