1 / 27

Research assessment for 2010-2014 at the Academy of Sciences of the Czech Republic Eva Zažímalová

The Academy of Sciences of the Czech Republic. Research assessment for 2010-2014 at the Academy of Sciences of the Czech Republic Eva Zažímalová. The Academy of Sciences of the Czech Republic. Aims Principles Organizational structure, disciplines

gavan
Download Presentation

Research assessment for 2010-2014 at the Academy of Sciences of the Czech Republic Eva Zažímalová

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Academy of Sciences of the Czech Republic Research assessment for 2010-2014 at the Academy of Sciences of the Czech RepublicEva Zažímalová

  2. The Academy of Sciences of the Czech Republic • Aims • Principles • Organizational structure, disciplines • Qualitative profile of teams and Institutes (Phase I) • Evaluation of teams and Institutes – other aspects (Phase II) • Time schedule Research Evaluation Seminar 2014, Prague, April 17, 2014

  3. The Academy of Sciences of the Czech Republic • Aims • Principles • Organizational structure, disciplines • Qualitative profile of teams and Institutes (Phase I) • Evaluation of teams and Institutes – other aspects (Phase II) • Time schedule Research Evaluation Seminar 2014, Prague, April 17, 2014

  4. The Academy of Sciences of the Czech Republic Aims • To obtain qualitative and quantitative information about science in the Academy of Sciences of the Czech Republic, in national, European and worldwide context, including differentiation of teams and Institutes according to quality. • To obtain complex information for strategy management of the Academy as a whole, including financing of the Institutes. • To mediate independent and comparable evaluation and feed-back for management of individual Institutes and research teams. • To meet the demands of responsibility in using public resources, including publishing the evaluation results. Research Evaluation Seminar 2014, Prague, April 17, 2014

  5. The Academy of Sciences of the Czech Republic • Aims • Principles • Organizationalstructure, disciplines • Qualitative profile ofteams and Institutes (Phase I) • Evaluationofteams and Institutes – otheraspects (Phase II) • Timeschedule ResearchEvaluationSeminar 2014, Prague, April 17, 2014

  6. The Academy of Sciences of the Czech Republic • Previousevalutation (2005-2009), performed in 2010 • 6th EvaluationexerciseattheAcademy • Increasingqualityofevaluationexercises, includingimpact(s) on theInstitutes • Firstevaluation on thelevelofresearchteams, but evaluationofInstitutes as a wholewasalsoincluded. • Evaluation on-site, based on peer review, internationalEvaluationCommitteesorganizedaccording to SectionswithintheResearchAreasoftheAcademy (9 sections – 9 Committees) ResearchEvaluationSeminar 2014, Prague, April 17, 2014 6/26

  7. The Academy of Sciences of the Czech Republic • Previousevalutation (2005-2009) - drawbacks and imperfections: • Definitionsofresearchteams – toodifferent in somecases • SteeringCommittee – unclearcompetences • Establishment ofEvaluationCommittees (conflictofinterest, insufficientknowledge re bibliometryetc.) • Establishment ofEvaluationCommitteesaccording to Sections (not according to disciplines) • UneaqualrigidnessbetweenEvaluationCommittees (comparisonbetweenSections not possible) • Insufficientknowledgeaboutinterpretationofscientometric data in Institutes, teams, and EvaluationCommittees (summof IF per capita, IF as themainindicatorofquality, underestimationoftrends, etc.) • Itwas not obligatoryforforeingevaluators to evaluateallteamswithinthe Institute (free choiceofonlysometeams) • Shortevaluation on-site (oneday) • Vaguedefinitionofrulesfor appeal/rebuttal ResearchEvaluationSeminar 2014, Prague, April 17, 2014 7/26

  8. The Academy of Sciences of the Czech Republic 5 Principles: 1. Evaluation within research disciplines 2. Informed peer review 3. Two-phase evaluation 4. Evaluation of scientific quality separated from financing 5. Transparentness Research Evaluation Seminar 2014, Prague, April 17, 2014

  9. The Academy of Sciences of the Czech Republic Principle 1 - Evaluation within research disciplines • Quality of outputs and results of teams will be evaluated within disciplines (associated into groups of disciplines) by Panels (phase I) and Review Committees (phase II) consisting of foreign evaluators. • Evaluation within disciplines will respect specific features of particular disciplines and will enable comparison with worldwide and European level of the disciplines. Research Evaluation Seminar 2014, Prague, April 17, 2014

  10. The Academy of Sciences of the Czech Republic Principle 2 – Informed peer review • Based on evaluation of quality of particular outputs and results submitted by individual researchers and teams (publications, citations, bibliometric data), and evaluation on-site. • Bibliometric evaluation: standard for natural sciences, not for humanities and social sciences. Research Evaluation Seminar 2014, Prague, April 17, 2014

  11. The Academy of Sciences of the Czech Republic Principle 3 - Two-phase evaluation • Phase I: Peer reviewofresearchoutputs and results – foreignevaluatorsorganizedintoPanels (remote, web-based). Criterion – quality in internationalcontext, regardlesstheInstitutes. • Phase II: Peer reviewofInstitutes – internationalReview Committees, evaluationresultsfromthePhase I willbesummarizedatInstitutes and integratedintothecontextofevaluationofotheraspectsofIntitutes’ activities. Evaluation on-site. Research Evaluation Seminar 2014, Prague, April 17, 2014

  12. The Academy of Sciences of the Czech Republic Principle 4 – Evaluation of scientific quality separatedfrom financing • Evaluation of scientific quality without direct relationship to financing. • Final report about the evaluation exercise 2010-2014 will not contain any financial advice and recommendations. • However: Information about utilization of financial resources in relation to financial demands of particular discipline and to scientific outputs and results will be one of the outputs of evaluation exercise. Research Evaluation Seminar 2014, Prague, April 17, 2014

  13. The Academy of Sciences of the Czech Republic Principle 5 – Transparentness • Preparation and arrangement: Institutes will be informed regularly. Trial registration to disciplines. • Publishing: Results of the evaluation exercise will be published in the frame of the Academy, and presented on the web-site of the Academy. Co-operation with PR-department. • Enhancement of quality of evaluation and operational communication with Institutes (in Phase II). Research Evaluation Seminar 2014, Prague, April 17, 2014

  14. The Academy of Sciences of the Czech Republic • Aims • Principles • Organizational structure, disciplines • Qualitative profile of teams and Institutes (Phase I) • Evaluation of teams and Institutes – other aspects (Phase II) • Time schedule Research Evaluation Seminar 2014, Prague, April 17, 2014

  15. The Academy of Sciences of the Czech Republic Organizational structureWithin the Institutes: Assessment units(each usually composed of one research team) Team member INSTITUTE Teams Team member – Researcher (categories 3 to 5 according to Career Structure of the Academy Research Evaluation Seminar 2014, Prague, April 17, 2014

  16. The Academy of Sciences of the Czech Republic Disciplines • Based on theofficialinternationallists(ERC, OECD, REF/RAE) in context to JCR (WoS) categories. • OECD („Revised field of science and technology (FOS) classification in the Frascatimanual”): 6 groupsofdisciplines / 42 disciplines fortheAcademy 5 groupsofdisciplines / 27 disciplines (based on trial registration): 1. Natural sciences (includingAgricultural sciences) 2. Engineering and technology 3. Medical and health sciences 4. Social sciences 5. Humanities Research Evaluation Seminar 2014, Prague, April 17, 2014

  17. The Academy of Sciences of the Czech Republic Disciplines – Natural sciences (example) Research Evaluation Seminar 2014, Prague, April 17, 2014

  18. The Academy of Sciences of the Czech Republic OECD List of disciplines and JCR categories Example: Discipline 1.4 Chemical sciences – sub-disciplines Research Evaluation Seminar 2014, Prague, April 17, 2014

  19. The Academy of Sciences of the Czech Republic The Academy: Disciplines and outputs Number of papers 2010-2013 (WoS) Natural sciences Počty publikací WoS v oborech 1 – 27 (2010-2013) Engineering& technology Multidisciplinary sciences Medical and health sciences Social sciences Humanities Research Evaluation Seminar 2014, Prague, April 17, 2014

  20. The Academy of Sciences of the Czech Republic • Aims • Principles • Organizational structure, disciplines • Qualitative profile of teams and Institutes (Phase I) • Evaluation of teams and Institutes – other aspects (Phase II) • Time schedule Research Evaluation Seminar 2014, Prague, April 17, 2014

  21. The Academy of Sciences of the Czech Republic Phase I: Qualitative profile of teams and Institutes • Each researcher: Maximum 4 scientific outputs • Team: The best result(s) (consisting of outputs) for a team (in relation to number of researchers in a team). • Remote evaluation • Qualitative profile will be generated (outputs and results assorted into 5 classes) Research Evaluation Seminar 2014, Prague, April 17, 2014

  22. The Academy of Sciences of the Czech Republic Evaluation: Data and requirements for evaluators • Bibliometryforallsubmittedoutputs • Alloutputsatthedisposal (web links), for team resultsallrelevantoutputs • Requirement: Evaluationofcontent in contextwiththediscipline Research Evaluation Seminar 2014, Prague, April 17, 2014

  23. The Academy of Sciences of the Czech Republic • Aims • Principles • Organizational structure, disciplines • Qualitative profile of teams and Institutes (Phase I) • Evaluation of teams and Institutes – other aspects (Phase II) • Time schedule Research Evaluation Seminar 2014, Prague, April 17, 2014 23/26

  24. The Academy of Sciences of the Czech Republic Phase II: Evaluation of teams and Institutes – other aspects • In Phase II, otheraspectsofteams and Instituteswillbeevaluated, such as: - grants, projects, collaborations - management (humanresources, careerpolicy) - education (teaching, supervisionofstudents) - otherscientificactivities (scientificinfrastructures, popularizationetc.) • The Institute and teamswillsubmitself-evaluation report • Evaluationon-site: International Review Committees Research Evaluation Seminar 2014, Prague, April 17, 2014

  25. The Academy of Sciences of the Czech Republic • Aims • Principles • Organizational structure, disciplines • Qualitative profile of teams and Institutes (Phase I) • Evaluation of teams and Institutes – other aspects (Phase II) • Time schedule Research Evaluation Seminar 2014, Prague, April 17, 2014

  26. The Academy of Sciences of the Czech Republic Time schedule - tentative Key dates Research Evaluation Seminar 2014, Prague, April 17, 2014

  27. The Academy of Sciences of the Czech Republic Acknowledgements: prof. Jana Musilová, prof. Jiří Chýla, prof. Petr Ráb Members of Working Group of the Council for Sciences Members of Working Group of the Academy Council Research Evaluation Seminar 2014, Prague, April 17, 2014

More Related