Dr. Sanjaya Lall

1. Dr. Sanjaya Lall Professor, Developmental Economics Oxford University, UK

2. Dr. Sanjaya Lall • Former Senior Economist, World Bank • Advisor, Technology Policy, Innovation, Competitiveness, Industrial Technology, Foreign Direct Investment, Trade, Monitoring & Evaluation Systems • Former Senior Research Officer, Institute of Economics and Statistics, Oxford

3. Monitoring and evaluation of innovation projects & programs Sanjaya Lall

4. Need for monitoring and evaluation (M&E) • All investment proposals need to be evaluated if they compete for resources • All projects and programs have to be monitored for effective implementation, meeting deadlines and ensuring discipline in spending, purchasing and so on • All projects and programs must be evaluated during and after completion to ensure that direct and social objectives are achieved

5. M&E is exceptionally difficult for innovation projects & programs • Highly speculative and uncertain nature of innovation results: problem exists even within enterprises in allocating R&D budgets • But public projects face other problems: • Public goods nature of many outputs • Non-measurable nature of benefits of programs to • Create inter-firm & inter-industry linkages • Strengthen linkages between industry and research institutions • Even more diffuse nature of stimulating ‘technology culture’, modernization etc.

6. Nature of M&E depends on nature of innovation project Stimulate enterprise R&D Upgrade S&T institutions Strengthen linkages of TIs with industry Fund technology based firms (VCs, etc.) Improve IPR regime/ implementation

7. Two major components • Benefits to the economy • Achievement of specific project objectives

8. Inherently qualitative and diffuse nature of innovation benefits gives rise to large number of possible M&E measures

9. Economic benefits: three levels of analysis • Trends at the general economy-wide or industrial level • Before and after project • International benchmarks • Analysis of enterprises and institutions covered by project • Comparisons of enterprises within project with those outside it

10. Economic benefits: measures • R&D spending • Patents, publications • Competitiveness (in technology products?) • Labour productivity (or TFP) • Spin-offs, ‘technopreneurs’ • Employment (total or skilled, mfg or R&D?) • International R&D collaboration • Technology balance of payments • Inward and outward FDI (technology based)

11. Other possible measures, more industry specific… • Quality standards and reject rates • Order to delivery times • Customer satisfaction ratings • Inter-firm collaboration • Cluster effects: local purchases, joint training and R&D, other links

12. Economic benefits: sources of data • General industrial, trade and R&D data within country • Benchmarks from other countries • Industrial census (with additional data on technology) • Questionnaires to firms & institutions within and outside project • Interviews with firms and institutions in project

13. Economic benefits: methodology • Benchmarking over time or across countries • Econometric analysis of enterprises within & outside project • Case studies

14. Project objectives: three components (for implementing agencies) • Inputs • Outputs • Outcomes

15. Inputs… • Extent to which implementation meets targets on schedule, e.g.: • Disbursement of project funds • Purchases of equipment • Construction of facilities • Other milestones

16. Outputs, depending on agency… • Speed of processing patent applications • Extent of metrology needs met, conformance to international standards • % of budget earned from industry (contracts or research) • Restructuring, international accreditation etc. of laboratories • No. of R&D contracts, commercialization, no. of publications (impact), patents • Number of technology loans, repayment rates • Venture capital funding: extent, success

17. Outcomes (broad economic impact)… • Part are already covered under analysis of economic benefits • More specific measures of performance by institutions and firms directly involved in project

18. Specific performance measures • Enterprises • Growth and nature of R&D • R&D management • Access to technology financing • Access to MSTQ services • Efficiency of IPR regime • Extent of interaction with technology institutions, universities, etc. • Export and productivity performance

19. R&D: Internal institutional culture, responsiveness to and links with industry, spin-offs MSTQ: Increased efficiency of services, effects on productivity, competitiveness, FDI VC: Stimulation of hi-tech start-ups, IPOs General: Improvement of R&D performance and creation of ‘technology culture’, use of new technologies in production & exports, attraction of MNC R&D facilities and contracts, outward FDI, ‘cluster’ effects. Specific performance measures: institutions

20. Who does M&E? • Project agencies provide information on project inputs and outputs • Data is collected by statistical & project agencies, questionnaires administered by independent researchers • Analysis carried out by independent researchers

21. Dr. Sanjaya Lall Professor, Developmental Economics Oxford University, UK