CONTENTS

1. A Novel Evaluation System of Public Research Institute in JapanInternational Workshop on the Evaluation of Publicly Funded ResearchBerlin, 26-27 September 2005Naoto KobayashiNational Institute of Advanced Industrial Science and Technology (AIST)

2. CONTENTS Missions and Principles of AIST Evaluation Policy in AIST Novel Evaluation System from the Viewpoint of Outcome Evaluation in the Future for the Publicly Funded Research Institutes

3. Missions of AIST: for all mankind and global consciousnessto our future The mission ofAIST is to generate innovative technologies for the new industries and to contribute to the economic development of our country and to the lives of peopleand , thus, finally to the sustainable global world!! Missions of AIST in association with the Ministry of Economy, Trade and Industry (METI) Interdisciplinary Researches to enable to plan long-range governmental policies attributing to the current and future needs of society Type-II Basic Research & Full Research Environment Energy Life Science Information Communication Resources Electronics Advanced Researches to explore broad spectra of research fields and to integrate multidisciplinary subjects Manufacturing Standard& Measurements Machine Nanotechnology Oceanography Material Geology Basic Researches to develop and maintain high standards of scientific and engineering research

4. President Senior Vice-President Vice-President Auditor Advisory Board Auditors’ Office General Administration Headquarter Safety & Environmental Protection Dept. Emeritus Councilor Research Coordinator Emeritus Advisor Planning Headquarter Fellow Evaluation Dept. Public Relations Dept. Technology Information Dept. Research Center (28) 〔Sites〕 AIST Hokkaido AIST Tohoku AIST Tsukuba AIST Tokyo AIST Chubu AIST Kansai AIST Chugoku AIST Shikoku AIST Kyushu Collaboration Dept. Research support departments AIST Innovations (TLO) Research Institute (21) Intellectual Property Dept. International Affairs Dept. Research Initiative(5) General Administration Dept. Tsukuba Advanced Computing Center Human Resource Dept. International Patent Organism Depository Research administrationdepartments AIST Innovation Center for Start-ups Financial Affairs Dept. Geo-information Center Research Facilities Dept. June, 2005 Metrology Management Center Researchers: ~2,500 Administrative staff: ~720 Budget: ~900M$

5. 産総研 Research Bases of AIST old AIST 15 Research Institutes 8 Institutes in Tsukuba 7 Institutes in other regions AIST Hokkaido AIST Tohoku In 2001 new AIST Merged into 1 Institute 9 Research Basis AIST Tsukuba AIST Kansai AIST Tokyo Waterfront AIST Chugoku Tokyo Headquarters AIST Kyushu AIST Chubu AIST Shikoku

6. Concept of Full Research in AIST Contribution to the Society Application Development Finding of New Knowledge Integration of Knowledge Theory Principle President Dr. Yoshikawa Economic & Social Needs Unknown Phenomena Theorem Type-I Basic Research Law Type-II Basic Research Coherent and concurrent research based on scenario Full Research AIST defines Type-II Basic Research, which integrates multiple disciplines and creates methods for the use of integrated knowledge; traditional basic research is defined as Type-I Basic Research, which is the pursuit and discovery of novel laws, principles and theories that govern natural phenomena. AIST places its highest priority on the pursuit of complete research, "Full Research", ranging from Type-I Basic Research to the development of “products” by conducting intensive Type-II Basic Research.

7. Sustainable Development by Full Research Society Outcomes AIST Public resources People’s Interest (applause) dream nightmare reality Universities Type-1 basic research Type-2 basic research Industrial development Industries AIST Full Research Innovation!!

8. Council for Science and Technology Policy R&D Evaluation Guideline (1997, 2001, and 2005）： ・appropriate and fair evaluation→ flexible, competitive, and open research environment ・encouraging evaluation→original and promising R&D, researchers with great ambitions and abilities , and significant policy and program ・announcement of results of evaluation→ accountability to the nation ・strict evaluation→ concentrated and effective allocation of resources ⇒ internationally high level of R&D which contributes to society and economy, and to development of new discipline Independent Administrative Institution Act2001 Introduction of evaluation system in AIST ・setting Evaluation Department ・annual evaluation2001～2004 AIST Evaluation System Advisory Committee 2003～5 How to evaluate R&D in AIST as an Independent Administrative Institution promoting R&D 　 ・auto-evaluation ・evaluation interval ・outcome Evaluation from the viewpoint of Outcome

9. Evaluation System of AIST Medium-Term Plan Reflection on Future Management Evaluation Committee for IAI in METI President Medium-Term Target Annual Report Report Evaluation Department Evaluation of Activity Results of Evaluation Comment for Management Evaluation Committees (Peers, Specialists or Intellectuals, Internal Reviewers) Advisory Board Research Resources Comment Evaluation of Each Unit’s Research Units (Center, Institute and Initiative)

10. Significanceof Evaluation • To evaluate research activities is to encourage researchers with great ambitions and abilities and to assist research organizations which contribute to the development of economy and the quality of our lives. • The top management of AIST requires the recognition of check and balance function in the evaluation of research activities, to consider the allocation of resources for research and to be equal to the occasion such as the reorganization. • AIST is also responsible for the accountability for public resources by showing performances and achievements to the society.

11. Novel Evaluation System of Research Units Purpose: • Activation and Increasing in Efficiency • Making sure the research strategy • Reference to the Managements Evaluation Intervals Startup Interim Summarized RI and RC Following RC Research Units ・Reconsideration of roadmap ・Output Indexes to outcomes ・Management procedure Roadmap toward outcomes The 1st Term (FY2001-2004) The 2nd Term (FY2005-2009) Based on Output Every year Toward Outcome Every a few years Change

12. Companies、New Startups Production・Marketing Pattern of Outcome Creation(Product Type) Solar Panel Startups Collaboration/ Technology Transfer (Prototype・Improvement） New Industry New Market Research in AIST Liquid Crystal Display Output Sample Market Outcome Market Share Novel Idea Transfer Acceptance in Market & Society

13. Pattern of Outcome Creation(Database Type) Extension of External Access αVersion，β Version Supply DB Open to the Public Organic Compounds Spectrum DB New Industry New Market System Adjust and Implementation in Startups and/or Companies Startups Collaborative Work DB Construction Acceptance in Domestic and International Society System Adjust and Implementation in AIST Research in AIST System DevelopmentSystem Improvement・Data Accumulation Transfer Market Share Sample Market Outcome Output Novel Idea Acceptance in Market & Society

14. Evaluation from the Viewpoint of the Outcome ①： Evaluation of Realized Outcomes ②： Evaluation of Scenario/Roadmap ③： Evaluation of Outputs to contribute to Outcomes ④： Evaluation of Management to contribute to Outcomes Past Present Research Future Outputs in the Past ① ： Realized Outcomes ②：Roadmaps ③：Outputs ④：Management Expected Outcomes in the Future Evaluation is performed through Items ②～ ④

15. Evaluationfrom the viewpoint of outcometo reflect the future

16. Procedure of Evaluation external reviewers: Specialists or Intellectuals Strategy of AIST internal reviewers: Principal Reviewers Research Units ＊Roadmap Evaluation ＊Output Evaluation ＊Management Evaluation Roadmap Evaluation ＊Roadmap for outcomes ＊Output indexes ＊Management Evaluation: Rating with comments Comments Resource Allocation Evaluation Department Reorganization Nation President of AIST Evaluation Committee

17. Evaluation of Roadmapto make sure the research strategy

18. Lamp Ballasts Example of Roadmap for Outcomes:Ultra-Low-Loss power devices （Power Electronics Research Center） 2002 20042006 2008201020122020 The 2nd Term The 1st Term Subjects (Targets) Substrate / Wafer Epitaxy Processing Packaging FET Devices Modules Applications 3 inch Wafer 4 inch Wafer 6 inch Wafer > Few 10μm/h Multi-Wafer-Epi. R&D in AIST low-Ron Bipolar-Tech. Self-Alignment High-reliable SiO2 Low-Temp Process T j>250℃ Wire-less （ＦＥＴ） （ＩＧＢＴ） Technology transfer 600V10A 2.5kV100A Bipolar Devices 600V100A Capacitor-less Low-EMI Heatproof Dispersed Power supply Power Transmission System Factory Motor drive Instruments General Inverter Air-Con. Electric Power Distribution System

19. Procedure of Evaluation internal reviewers: Principal Reviewers external reviewers: Specialists or Intellectuals Strategy of AIST Research Units ＊Roadmap Evaluation ＊Output Evaluation ＊Management Evaluation ＊Roadmap for outcomes ＊Output indexes ＊Management Output Evaluation Evaluation: Rating with comments Comments Resource Allocation Evaluation Department Reorganization Nation President of AIST Evaluation Committee

20. Evaluation of OutputIndexesto know the level of research unit

21. Procedure of Evaluation internal reviewers: Principal Reviewers external reviewers: Specialists or Intellectuals Strategy of AIST Research Units ＊Roadmap Evaluation ＊Output Evaluation ＊Management Evaluation ＊Roadmap for outcomes ＊Output indexes ＊Management Management Evaluation Evaluation: Rating with comments Comments Resource Allocation Evaluation Department Reorganization Nation President of AIST Evaluation Committee

22. Evaluation of Management Concept of Full Research : Policy statement

23. Procedure of Evaluation external reviewers: Specialists or Intellectuals internal reviewers: Principal Reviewers Strategy of AIST Research Units ＊Roadmap Evaluation ＊Output Evaluation ＊Management Evaluation ＊Roadmap for outcomes ＊Output indexes ＊Management Evaluation: Rating with comments Comments Resource Allocation Evaluation Department Reorganization Nation President of AIST Evaluation Committee Nation Evaluation Committee

24. Novel Evaluation System of Research Units Purpose: • Activation and Increasing in Efficiency • Making sure the research strategy • Reference to the Managements Evaluation Intervals Startup Interim Summarized RI and RC Following RC Research Units ・Reconsideration of roadmap ・Output Indexes to outcomes ・Management procedure Roadmap toward outcomes The 1st Term (FY2001-2004) The 2nd Term (FY2005-2009) Based on Output Every year Toward Outcome Every a few Years Change

25. Mid-term Objectives by the minister (e.g. METI) 3-5 years Mid-term Plans by each Institute Check of Performances & Achievements Evaluation Committee in each ministry Evaluation Committee in Ministry of Internal Affairs and Communications

26. President Senior Vice-President Vice-President Auditor Advisory Board Auditors’ Office General Administration Headquarter Safety & Environmental Protection Dept. Emeritus Councilor Research Coordinator Emeritus Advisor Planning Headquarter Fellow Evaluation Dept. Public Relations Dept. Technology Information Dept. Research Center (28) 〔Sites〕 AIST Hokkaido AIST Tohoku AIST Tsukuba AIST Tokyo AIST Chubu AIST Kansai AIST Chugoku AIST Shikoku AIST Kyushu Collaboration Dept. Research support departments AIST Innovations (TLO) Research Institute (21) Intellectual Property Dept. International Affairs Dept. Research Initiative(5) General Administration Dept. Tsukuba Advanced Computing Center Human Resource Dept. International Patent Organism Depository Research administrationdepartments AIST Innovation Center for Start-ups Financial Affairs Dept. Geo-information Center Research Facilities Dept. June, 2005 Metrology Management Center Researchers: ~2,500 Administrative staff: ~720 Budget: ~900M$

27. Improvement in services Increase in efficiency Activation of operation Evaluation of Research Administration Departments and Research Support Departments Aim: Support for research activities of AIST More effective execution of administrative and supportive work ・Setting of “Proposal Box” for accepting opinions to improve administrative work ・List-up and sorting of the importance of work ・Re-arrangement of the staffs so as to maximize the total efficiency of the work ・Quantitative target-setup ・Preparation of “Work Flow Manual”

28. Strategy of Evaluation Evaluation of R&D Policy Strategy Plan Do Implementation Evaluation See Evaluation of R&D in the Research Institute Action Management Check Report Evaluation Planning Proposal Plan Advice Output Input Research Do

29. What is Innovation? ◆Adequate Evolution of the Economy, Societyand Environment based on Novel Science & Technology The trend to intensify the interface of Science and Industrial Innovation is prevailing at universities and public research industries in the world. Especially it is necessary to make breakthroughs by technological fusion in the interdisciplinary fields (ICT, Nanotechnology, Biotechnology etc.).

30. Conclusion: Evaluation for Innovation R&D Industrial Society Innovation!! • Industrial Transformation is necessary toward the Sustainable Global Society. • Innovation on the Basis of Novel R&D is necessary for the Industrial Transformation. • Adequate Evaluation System of R&D which contributing to the Innovation is important, especially for the Publicly Funded Research Institutes. • Evaluation from the Viewpoint of the Outcome is one of the attractive tools for the novel evaluation.

31. Evaluation is for Encouraging with Love. Evaluation is for Communication. Evaluation is for Strategy. Evaluation is for Creation. Evaluation is to be fed back. Evaluation reflects the Future. Notes: Evaluation is like a Compass for Voyage.

32. Thank you !

33. Appendix

34. Definition of independent administrative institution 【Performance】 - to behighly public - neednot to be performed by government directly - hard to be achieved by private organization independently ① effective and with highquality ② independent ③ open

35. - to set mid-term objectives and to receive periodical evaluation by third party - to entrust some service to private organization or to abolish institutions - to manage as a private corporation - to organize minimum size ① To performeffectivelyandqualitatively

36. to concentrate authority to president to exclude the excess involvement of minister to have an appropriate personnel including executives from private corporation ② To performindependently ③ To performopenly - to open all performances and achievements to the public

37. Evaluation Committee 【outside of AIST】 【inside of AIST】

38. 研究センター Middle Evaluation Terminal Evaluation Middle Evaluation Start up Research Center ・top down ・time limit 研究部門 ・産総研の看板 Research Institution ・bottom up ・no time limit ・技術ポテンシャルの発展 ・時限的 Advise for Roadmap ・継続的 Middle Evaluation Terminal Evaluation Advise for Roadmap 研究ラボ Middle Evaluation Research Initiative ・to be Research Center ・time limit ・研究センターの創出 ・時限的 Advise for Roadmap A newResearch Center Evaluation Interval in the 2nd Research Phase

39. Reorganization by Unification History of AIST ～ Organizational shift in accordance with society needs at the times～ National Institute of Advanced Industrial Science and Technology Aiming at COE Aiming at Front Runner Recovery from World War II Catching up advanced nations 1970 1990 1948 1980 2000 1960 Establishment of Former AIST 1903 Weighing & Testing National Research Laboratory of Metrology Mechanical Engineering Laboratory Machinery Tokyo National Research Laboratory Chemical Technology National Institute of Materials and Chemical Research 1900 Alcohol Microorganic Industry National Institute of Bioscience and Human-Technology Fermentation Textile Fiber & Polymer Nat’l Inst.for Advanced Interdisciplinary Research Production Science Industrial Arts 1882 Geological Survey of Japan Geological Survey Electronics Electrotechnical Laboratory Electrical 1891 Resources & Pollution National Institute for Resources and Environment Mining Resources 7 Regional National Industrial Research Institutes 2 Regional Institutes 7 Regional Institutes Weights and Measures Training Institute Concept for Reorganization Merging research fields and Administrative needs Industry Type Research Areas