1 / 18

Activities of the ESF Member Organisation Forum on Research Evaluation

Activities of the ESF Member Organisation Forum on Research Evaluation. Beispielfolie Inhaltsverzeichnis (mit Bild). ESF Member Fora Structure of the presentation. 1. Objective of ESF Member Fora ESF Member Forum on Ex-Post Evaluation of Funding Schemes and Research Programmes

hop-green
Download Presentation

Activities of the ESF Member Organisation Forum on Research Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Activities of the ESF Member Organisation Forum on Research Evaluation

  2. BeispielfolieInhaltsverzeichnis (mit Bild) ESF Member ForaStructure of the presentation 1. Objectiveof ESF Member Fora • ESF Member Forum on Ex-Post Evaluation ofFundingSchemesand Research Programmes • The new Forum on Evaluation ofPubliclyFunded Research

  3. BeispielfolieInhaltsverzeichnis (mit Bild) ESF Member ForaStructure of the presentation 1. Objectiveof ESF Member Fora • ESF Member Forum on Ex-Post Evaluation ofFundingSchemesand Research Programmes • The new Forum on Evaluation ofPubliclyFunded Research

  4. ESF Member ForaObjective of the instrument The instrument “ESF Member Organisation FORUM” was developed in response to clearly expressed needs from MOs in consultations on the Strategic Plan 2006-2010 Quoting the Strategic Plan: “MO Fora is an output-oriented, issue-related venue for MOs, involving others as appropriate, to exchange information and experiences and develop joint actions in science policy”. “ESF aims to be the actor of choice and preferred partner when national research organisations develop joint activities.”

  5. Completed, current and new Member Fora • Completed Member Fora • Promoting Internationalisation of Social Sciences in Central and Eastern Europe (6/2006-12/2008) • Evaluation of Funding Schemes and Research Programmes (10/2007-11/2009) • Research Careers (11/2007-4/2010) • Current Member Fora • Peer Review (10/2007-12/2011, extended) • Research Integrity (9/2008-12/2010) • New Member Fora • Medium Sized Research Infrastructures (1/2010-2011) • Science in Society Relationships (1/2010-9/2012) • Evaluation of publicly funded Research (4/2010-1/2012) • Indicators of Internationalisation (11/2009-11/2010) • Research Careers (...)

  6. BeispielfolieInhaltsverzeichnis (mit Bild) ESF Member ForaStructure of the presentation 1. Objectiveof ESF Member Fora • ESF Member Forum on Ex-Post Evaluation ofFundingSchemesand Research Programmes • The new Forum on Evaluation ofPubliclyFunded Research

  7. ESF MOF on Evaluation of Funding Schemes and Research Programmes – Aims • Focus: “Post-Grant” Evaluation i.e. if the funding schemes or the research programmes achieve their stated aims. • Objectives • exchange and document experiences with current practices • facilitate networking of science officers engaged in evaluation • explore needs and possibilities for collaboration in future evaluation exercises. • Participation: 38 Organisations

  8. ESF MOF on Evaluation of Funding Schemes and Research Programmes – Workshops 1st Workshop • Evaluation of Funding Schemes and Research Programmes: Expectations, Practices and Experiences. Berlin, 22.-23. October 2007 (Host: DFG) 2nd Workshop • Quantitative Indicators in ex-post Evaluation of Funding Schemes and Research Programmes: Case studies and Rationales. Rome, 31. March -1. April 2008 (Host: INFN) 3rd Workshop • Quality of Quality Assurance. Striving for Highest Standards in ex-post Evaluation. Vienna, 6.-7. October 2008 (Host: FWF) 4th Workshop • Evaluating socio‐economic impact of research funding schemes. Budapest, 27.-28. April 2009 (Host: OTKA) 5th Workshop • Evaluation Strategies and Comparative Evaluation Techniques. Dublin, 16.-17. November 2009 (Hosts: HRB and SFI)

  9. ESF MOF on Evaluation of Funding Schemes and Research Programmes – Outcome • Action Items • Mapping study on evaluation of funding schemes –finalised • A pilot study on international benchmarks – will continue in a new forum • Final Outcome • Report: Evaluation in National Research Funding Agencies: approaches, experiences and case studies can be downloaded: http://www.esf.org/activities/mo-fora/publications.html

  10. BeispielfolieInhaltsverzeichnis (mit Bild) ESF Member ForaStructure of the presentation 1. Objectiveof ESF Member Fora • ESF Member Forum on Ex-Post Evaluation ofFundingSchemesand Research Programmes • The new Forum on Evaluation ofPubliclyFunded Research

  11. ESF MO Forum Future • New Forum on Evaluation of Publicly funded Research, approved by ESF Governing Council • Second Forum „Indicators of Internationalisation“ • Proposing Member Organisations: • DFG, Germany; HRB, Ireland; NOW, Netherlands; FWF, Austria; FNR, Luxembourg; The Research Council of Norway; Swedish Research Council; Hungarian Scientific Research Fund. • Towards good practices and joint evaluation exercises • Alignment with EUROHORCs/ESF Road Map: • Chapter 6. Develop common approaches to ex-post evaluation of funding schemes and research programmes • Strengthen Workshop format

  12. ESF MO Forum on Publicly Funded Research • First meeting 10th to 11th May 2010 in Stockholm • three more meetings within the next two years (next meeting Nov. 2010) • set up three working groups that drafted their work plan: • Assessing the Impact of Funding Schemes on Science and Society • Ensuring and Improving quality in ex-post evaluation of Funding Schemes and Research Programmes • Classification systems and Categorization of output data  • Envisaged outputs: • a web-based repository of recent evaluation studies • guidelines for evaluation – commenting on state of the art • mapping the evaluation practices/the evaluation function in funding organisations • overview of impact studies and identification of gaps • recommendation on possible design of impact studies • a pilot study on the classification and collection of output data • a document on steps and pitfalls on the way towards a common classification

  13. Working Group on Classification systems and Categorization of output data  • Examine feasibility pilot for Health Research Classification System (HRCS) coding of portfolios in FNRS and DFG • Final reports and data for statistical and evaluative purposes: Structuring final reports helps to gather data that allow to answer statistical questions on topics like • male/female scientific staff in funded projects • nationality of scientific staff-members (indicates internationality) • age of staff-members (indicates „age of a field“) • collaboration with national and international partners (by country) (indicates cluster and networking) • Output (publications, patents, articles in proceedings etc.) • Qualifications (dissertations of staff members)

  14. Final Reports: Method for achiving structured informations in the DFG • DFG-Guidelinesnamethetopicstobementioned in a final report • DFG planstocollectthesedata via a PDF-basedstandard-form • Data withintheseforms will beextractedautomaticallyandtransferredinto a database • Someofthesedataarepublished via DFG-Internet (e.g. thepublicationlisthttp://gepris.dfg.de ) • Others will beusedonlyforstatisticalpurposes (e.g. ageandnationalityofscientificstaffmembers) • Project Status DFG: • Conceptualconsiderations on howtoandwhichdatatocollectarefinished • Considerations on technical solutions (e.g. transfertodatabase) and legal framework in final stage • In thebeginningof 2011 thestandardizedcollectionof final reportsof individual grants will beimplemented

  15. Method for achiving structured informations in the FWF (Austrian Science Fund) • Collection of project output data in the FWF Source: Collecting and using project output data at the FWF, Rudolf Novak, FWF, Austria 

  16. Method for achiving structured informations in the FWF (Austrian Science Fund) - Publications Source: Collecting and using project output data at the FWF, Rudolf Novak, FWF, Austria  ca. 10.000 Scientific Journals

  17. Working Group on Classification systems and Categorization of output data  • Next Steps: • Produce short paper summarising approaches for the analysis of output data • Circulate/share details of work ongoing in each research organisation which involves the gathering of information about research outputs/outcomes and impacts. • Generate recommended “core” set of outputs that could/should be collected by research organisations, plus guidance on definitions etc. • Plan pilot exercise in which data for “core” outputs is brought together across a number of research organisations. • Discuss the results of each step in the whole MO Forum • Contribute to the comparability of Research Funding!

  18. Thank you for your attention! for further questions please contact: Anke Reinhardt Deutsche Forschungsgemeinschaft (DFG) -Informationsmanagement- D-53170 Bonn Tel. +49 (228) 885-2588 Fax +49 (228) 885-2777 Anke.Reinhardt@dfg.de http://www.dfg.de Laura Marin Science Officer MOs Relations and Partnerships Corporate Science Strategy Development European Science Foundation Tel.: +33 388 76 7138 lmarin@esf.org http://www.esf.org

More Related