1 / 20

December 12, 2013, Zagreb

Evaluation culture in SEE (Public procurement in SEE innovation evaluations: A comparative and needs assessment study; Lena Tsipouri – Nikos Sidiropoulos University of Athens, Centre of Financial Studies. December 12, 2013, Zagreb. Outline.

keegan-moody
Download Presentation

December 12, 2013, Zagreb

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation culture in SEE (Public procurement in SEE innovation evaluations: A comparative and needs assessment study; Lena Tsipouri – Nikos Sidiropoulos University of Athens, Centre of Financial Studies December 12, 2013, Zagreb

  2. Outline • Does a comparative study and benchmarking make sense (objective/subjective indicators)? • Methodological remarks (literature; research) • Experiences from the EVAL INNO countries • A Synthesis • Lessons from intelligent benchmarking

  3. Does a comparative study and benchmarking make sense • The background of the countries studied differs considerably (Austria-Greece-Hungary-Bulgaria / Montenegro-Serbia) • The legal framework is hardly applicable to the size of the procurement studied • Objective indicators (measurable such as number of evaluations; budgets; frequency, but also qualitative regarding quality and response) are limited • Subjective indicators can be obtained but are subject to criticism (the solution is stakeholder approval)

  4. Methodological remarks (literature) • Common rules for public procurement, as a tool of the Single Market (contracts over 499000; Open procedures, Restricted procedures, Negotiated procedures and Competitive dialogue). Usually RTDI Evaluations fall in the budget range of 50000-500000 Euros • There is no real academic literature; There is a broad number of tender documents and terms of reference available, mostly (but not exclusively) originating in procurement by the European Commission; • Information of the later stages of the procurement, namely monitoring of the evaluations and their acceptance, is practically absent;

  5. Methodological remarks (research) • Borrow notions for benchmarking from the pverall literature on public procuremnt • Quality provision of the service • Knowledge of the international standards and good practices • Knowledge of the local needs and capabilities • Knowledge of international capabilities and interest to respond to national tenders (price and reputational issues) • Good and timely decisions

  6. Public procurement of RTDI evaluations: our approach • Decompose the process: • Identifying the requirements and user readiness • Market intelligence • Tendering process (Terms of reference: background, data availability, questions and methods) • Assessing tenders and awarding contracts • Managing contract delivery • Response to recommendations

  7. The basic dimensions • The institutional set up (formal and informal rules) • Key organisations involved • Tendering process

  8. The institutional set up (formal rules) • The budget thresholds for general provisions for public tendering • The existence (or not) of special provisions for RTDI evaluations (e.g. specific thresholds; individual selection procedures etc.) • Explicit legislation (or not) regarding the legal obligation of awarding authorities to evaluate their programmes or organisations. • The existence (or not) of evaluation standards

  9. The institutional set up (informal rules) • Relevant parameters for launching tenders (strategy issues): • Frequency • Type of evaluations

  10. Key organisations involved • Awarding Authorities (how many, how good, how can they improve) • Evaluators (local, national, international; issues of independence, expertise and reliability for evaluators called for direct or restricted tenders; how good, how can the market evolve) • Other stakeholders (exercise pressure for RTDI evaluations)

  11. Tendering process • Terms of Reference (how good they are/could be) • Smooth process (no legal or other complications) • Time to contract (benchmarks) • Monitoring (hands on or off?) • Content (how ambitious are the Terms of Reference?) • Adoption of recommendations (of the specific evaluation and more in general)

  12. Type of parameters

  13. Type of parameters

  14. Experiences from the EVALL-INNO countries • Measuring/subjective rating per parameter decomposed • Comments per parameter • Comments per country

  15. An example of experiences from the EVALL-INNO

  16. Experiences from the EVALL-INNO countries • The European Commission defined PRAG – Practical Guide to Contract Procedures for EU External Actions

  17. Experiences from the EVALL-INNO countries

  18. A topic for benchmarking

  19. What next: • Do countries want to incorporate the benchmarking lessons (what should be their priorities; what is their distance to top; who are the top performers; from whom to learn)? • Should/could the EU play a role in taking the (obvious but documented) results a step further? If yes how? • Are other stakeholders interested in the results (to be read horizontally or vertically)

  20. tsipouri@econ.uoa.gr nikos.sidiropoulos@gmail.com

More Related