170 likes | 298 Views
Reformation of National R&D Program Evaluation System. Woo Chul Chai. I. Overview of S&T in Korea. I. I. I. I. I. Contents. National R&D Program Evaluation System. Concluding Remarks. 2. Overview of S&T in KOREA. 3. Changes of S&T Environments in Korea. 1960s. 1970s. 1980s.
E N D
Reformation of National R&D Program Evaluation System Woo ChulChai
I Overview of S&T in Korea I I I I I Contents National R&D Program Evaluation System Concluding Remarks 2
Changes of S&T Environments in Korea 1960s 1970s 1980s 1990s 2000s Policy trend Technology- oriented policy Industry- oriented policy Enhancing technology innovation Building R&D infrastructure Promoting R&D GRIs established (e.g. KIST) MOST established in 1967 Daedeok Science Town built in 1974 Enforcing the national R&D programs Promoting the university -based researches (e.g. SRC, ERC) S&T policy direction Increasing efficiency of R&D investment (coordination of S&T- related policies) Planning Total Roadmap Heavy industry goods & electronic products Light & heavy industry goods Change in focusing industry Light industry goods Electronic & transport products Primary goods 4
Brief History of National R&D Programs Year New Growth Engine Program MOST National R&D Program MOCIE Industrial Technology Program MKE MIC Information & Communication Technology Program ’08 MOHW MEST Health & Medical Technology Program ME Environmental Technology Program MAF Agricultural Technology Program MOCT Construction & Transportation Technology Program Academic Research Promotion Program MOE 5
Science Competitiveness Technology competitiveness S&T Development in Korea (Quantitative Growth) 2th 0 5th 6th 6th 5 8th 12th 7th 12th 10 14th 14th 15th 16th World ranks 15 17th 19th 21th 20 25 27th 30 2001 2002 2003 2004 2005 2007 2008 2006 Source : IMD reports 6
History of National R&D Program Evaluation Increasing Needs of Effectiveness of R&D Investment Performance Management& Evaluation Enhancement of Effectiveness, Efficiency & Accountability Performance Evaluation Transformational Stage Survey· Analysis· Evaluation ○ ’08~ ○ Law on Performance Evaluation and Performance Management of National R&D Program Revised(‘08.2) ○ Focus : Performance Evaluation & Management for Enhancing Effectiveness of R&D Investment Effectiveness of R&D Investment Growing Stage ○ ’06~’07 ○ Law on Performance Evaluation and Performance Management of National R&D Program Enacted(‘05.12) ○ Focus : Performance Achievement based on the Results of R&D Activities Entering Stage • ○ ’97~’05 • ○ S&T Framework Law • Enacted(’01.1) • ○ Focus : Appropriateness • of Input and Output 8
Architecture of Program Evaluation in NES NSTC Review of Plan Reports of Results Specific Evaluation Meta Evaluation MOSF • Basic Plan for Performance Evaluation (5 yr) • Action Plan for R&D Evaluation (annually) • To evaluate the appropriateness of Self-Evaluation • To review evaluation procedure and methods of Self-Evaluation • In-depth evaluation on major national R&D programs • Long-term/large-scale programs • Joint program among ministries • Programs which need to remove redundancy and require connectivity • Programs at national issue • Mainly examined by the Evaluation Committee Evaluation Results Evaluation Guideline Self Evaluation Ministries • Planning Self-Evaluation according to MOSF’s guideline • Implementing Self-Evaluation • Annually practiced by Ministries • Implementation based on self-made performance indicators and methods 9
Lifecycle of National R&D Program Evaluation Evaluation Range Feasibility Analysis Self- Eval Meta-Eval Specific (In-Depth) Eval Self- Eval Meta-Eval Self- Eval Meta-Eval Follow-up Eval +3year +3year + 1year Time Start year Closing year Implementation Ex-ante Ex-Post 10
Self/Meta-Evaluation of National R&D Programs (1) Designed after PART (Program Assessment Rating Tool)of the U.S. Federal Government Line Ministriesassess their own programs every 3 years The assessment is based on 25 checklist (questions)for all types of programs - Types of programs: R&D (Basic, Applied Research, Development), Infrastructure Investment, Procurement of Large-scale Facilities and Equipment, Human Resource Development MOSF reviews the assessment results and reflects them in annual draft budgets and the National Fiscal Management Plan 11
Self/Meta-Evaluation of National R&D Programs (2) • Answers to the questions take the form of “Yes (5)” or “No (0) • - In case of the questions regarding the achievement of • program goals, 2-scale answers (5, 0) is given. • A different score is assigned to each question and the result of • assessment is given by the sum of score • - Classified as “Effective (95-100),”“Moderately Effective (90-94),”Adequate (75-89),” and “Not Effective (0-74).” 12
Limitations Little enthusiasm from line ministries Line ministries did not set upa clear framework of mission and strategy Performance indicators were not derived from ministerial missions in a systematic way Assessment of performance relies on subjective assessment by outside experts and in-house staff, not systematically utilizing indicators Cooperation and coordination among players are not sufficiently made 15
Future Works for Evaluation Developing Strategic Performance Management Frame - Reviewing Performance Indicators, Monitoring R&D Activities, etc. Enhancing Education & Consulting for R&D Program Officers - Organizing Education Program of Performance Management - Developing Logic Model Manual for Various R&D Program Types Linking Evaluation Result with Budget Allocation 16
Thank You!! Chai, Woo Chul wcchai@kistep.re.kr