1 / 20

new frontiers in evaluation Evaluating the RTD policy portfolio the Austrian experience

new frontiers in evaluation Evaluating the RTD policy portfolio the Austrian experience. Leonhard Jörg Andreas Schibany 24. April 2006. Road map. Why portfolio evaluation? Basic dimensions for evaluating RTD policy portfolios Some observations from Austria

erelah
Download Presentation

new frontiers in evaluation Evaluating the RTD policy portfolio the Austrian experience

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. new frontiers in evaluationEvaluating the RTD policy portfoliothe Austrian experience Leonhard Jörg Andreas Schibany 24. April 2006

  2. Road map • Why portfolio evaluation? • Basic dimensions for evaluating RTD policy portfolios • Some observations from Austria • Limitations and practical problems

  3. Why should we look more systematically on RTD policy portfolios? ... without a portfolio manager ... as long budgets keep expanding • End of catching up process is in sight • Attention may shift again from “how much we spend” to “how we spend” • There might be quite some room for increasing the effectiveness of the funding system

  4. Some remarks on the context • Portfolios are not designed on the drawing table but the result of • Changing perceptions of needs and problems • Changing ways of how R&D is undertaken (mode 1  mode 2) • Policy making in competitive environments • There is no optimal portfolio • Portfolios are usually messy with single instruments addressing multiple goals • We are looking after improvements rather than for THE optimal portfolio

  5. Road map • Why portfolio evaluation? • Basic dimensions for evaluating RTD policy portfolios • Some observations from Austria • Limitations and practical problems

  6. Basic dimensions for evaluating RTD policy portfolios (i) • Coverage: • What policy goals are covered? • Are there gaps? • Proportions: • Follow the money: How do the financial proportions fit to the policy agenda? • Follow the debate: Does the amount of attention devoted to single instruments correspond with „importance“

  7. Basic dimensions for describing RTD policy portfolios (ii) • Appearance/Visibility: • Are differentiations between neighbouring instruments/brands clear to the clients? • How many brands does the funding system communicate? • Take the perspective of beneficiaries/clients: • How many schemes/ programmes are available for specific RTD activities of specific groups: One? More than one? None? • Patterns of usage: • What instruments are used in parallel? • Are there migration patterns between instruments?

  8. What indicates quality? • Overall R&D-performance of the innovation system (hopefully) • Responsiveness to changing environments and needs • Interrelation between instruments (supporting complimentarity vs. interference and overlapping/competition) • Interrelation between different levels of RTD-policy (regional, national, international) • Entry rules and conditions for new instruments/programmes • Exit strategies

  9. Road map • Why portfolio evaluation? • Basic dimensions for evaluating RTD policy portfolios • Some observations from Austria • Limitations and practical problems

  10. Growing budgets

  11. Catching-up GERD/GDP

  12. Expanding policy portfolio Funding of institutions (universities, CRO’s) bottom-up project funding (ERP, FWF, FFF) first thematic programmes (energy) run by ministries Soft measures (coaching, information, IPR) more thematic programmes (transport, Flex-Cim,..) fiscal measures programmes … programmes Kplus, Kind/net, Fhplus, NW, NANO ... Research infrastructure, investments education Industry structure Critical masses Technology centres high-tech sectors excellence diffusion leverage effects clusters science-industry linkages

  13. Bottom-up project funding Institutional funding (colour of funding ministry) Programme funding Catalytic financial measures fiscal measures Committee for science, industry and economic affairs Parliament ERP Fund Government Policy Austrian Science Council BMF BMWA BMVIT BMBWK National Research Fund Anniversary Fund FFG Programmes / Agencies Structural Programmes Thematic Programmes Mobility/ Talent Research projects ARC Universities CD-L. R&D-projects Start-up, IPR, PE/VC Performers LB-S KFI AoS Firms Polyt.

  14. Financial and Fiscal Measures: Objectives and Instruments

  15. Financial Resources for main funding instruments

  16. Focus: direct funding

  17. Observations on the Austrian policy portfolio • High level of diversification • Strong in mobilising communities • Significant improvements in management and evaluation standards • Fragmentation – Tendency for establishing new programmes for ever smaller target groups • Increasing competition between programmes – competing for beneficiaries • Lack of portfolio management

  18. Road map • Why portfolio evaluation? • Basic dimensions for evaluating RTD policy portfolios • Some observations from Austria • Limitations and practical problems

  19. Limitations and practical problems • International benchmarking: • New collections of “good practice” examples usually remain vague on the portfolio side “it’s the recipe not the ingredients”) • Information base is dispersed and messy: • Monitoring routines at programme level can rarely be combined/matched • Evaluations on programme level usually address question of external coherence. However the big picture remains a patchwork • Where is the customer?

  20. Thank you for your attention !

More Related