The Programme for Research in Third Level Institutions (PRTLI) : An Irish Experience

1. The Programme for Research in Third Level Institutions (PRTLI): An Irish Experience

2. Presentation • Focus of presentation • Brief description of PRTLI • Key Guiding Principles for recent PRTLI Impact Assessment • The Impact Assessment Process • Closing Remarks

3. Ex-Ante Monitoring Ex-Post Ex-Ante Monitoring Ex-Post Monitoring Project level Programme Level Research System Evaluation

4. Objectives

5. To enable a strategic and planned approach by third-level institutions to the long-term development of their research capabilities..., • so as to enhance the quality and relevance of graduate output and skills….. • to encourage co-operation between researchers both within the institutions and between institutions….

6. SCALE / RESOURCES; €605M PROGRAMMES (€202m) INFRASTRUCTURE (€403m) ALL DISCIPLINES ENHANCED EDUCATION PRTLI DISTINGUISHING FEATURES (NDP funded) INSTITUTIONAL STRATEGY FOCUS COUNTERPART FUNDING EVALUATION

7. Addressing National Sectoral Needs% Total Allocation/Euros funding to date • Biosciences/Biomedical 48% 294m • (INCL. Food and Health Science, & Neuroscience) • Platform Technologies & Materials 14% 87m • (INCL. Chemical & Physical sciences) • Environment/Marine 11% 63m • ICT/Advanced Communications 10% 60m • Social Science/Humanities 8% 46m • Research Resources/Libraries 9% 54m

8. TO DATE….. • Funding of >60 research programmes • 17Research Centres & 1 national library completed 33 Centres/Institutes to be established • 99, 000m2 new purpose-built research space • Human capital- approx. 450 postdocs • - approx. 1000 postgrad. students

10. Monitoring and Evaluation

11. PROCESSES PRIORITISATION STRATEGIC PLANNING SELECTION SPILL-OVER INTER-INSTITUTIONAL CO-OPERATION OUTCOMES - FOR INSTITUTIONS TEACHING RESEARCH LEARNING CHALLENGE LEADERSHIP RESPONSE

12. QUANTUM RESEARCH ACTIVITY INCREASE STRENGTHENED OUTCOMES - NATIONALLY RESEARCH CENTRES NEW MULTI-DISCIPLINARY RESEARCH HUMAN CAPITAL

13. Independent International PRTLI Impact Assessment • Additional to ‘routine’ monitoring • Initiated October 2003 • Five years post programme launch • Approx. 4 years since initiation of flow • of funds

14. Informing Principles for Selection of Steering Committee • Independence of Steering Committee • International • Experience • Reputation • Composition of Steering Committee • In broad terms, representation of disciplines • Geographical balance • Gender balance

15. Steering Committee Composition Prof. E. Banda Secretary General ESF Former Secretary General for Universities and Research, Spain Prof. R. Vihko President of the Academy of Finland Prof. J. Morrill British & Irish History, Cambridge UK. Arts and Humanities Council Prof. L. Resnick Prof. Of Psychology, Director of the Institute for Learning, Univ.of Pittsburgh, USA.

16. Process • Independence • No engagement in evaluation by HEA Executive or Authority • Appropriate Tools • Diversity of tools • Facilitation of engagement with all stakeholders • Horizontal and vertical probing • Real and Meaningful Indicators • Programme characteristics • Stage of life cycle • Stakeholder diversity….requirements • ‘Collectable’ and cost effective

17. Policy Progress Management To address the broader policy issues including role of PRTLI in funding of research core capacity To assess the progress of the programmes funded to date To ascertain whether the objectives of the PRTLI scheme are being met To Capture implications for future programmes To Capture implications for wider society To monitor the level of adherence to documented plans To assess the administration of the programme by the HEA executive Objectives of Assessment

18. Scopeof Assessment • Commencement of the PRTLI scheme - to end December 2003 • Snapshot at a point in time in a dynamic process … • The heterogeneity of programmes taken into account…. • The Assessment to include many elements of final review of Cycle 1….

19. Collaboration Policy Review Institutional Strategy & Management Members of SC Teaching & Learning Visiting Experts Specialists In T&L Independent Secretary/ Rapporteur Steering Committee Research Visiting Experts Researchers Peer review Bibliometrics Support from CIRCA Europe Ltd (Bibliometrics CWTS/Van Raan Leiden) Support from Indecon International Economic Consultants

20. Research &Teaching/Learning • Six-Monthly Report Analysis • All reports for all programmes – by Research Consultants • Site Visits • 19 sites/~30 programmes • 3 international experts per site visit • Tours of facilites • Interviews: Centre/Programme Director; Heads of Department(s); Tenured academic staff ; post-doc researchers; post-grad students; undergraduate students

21. Bibliometrics CWTS (Leiden, Prof. Van Raan) Comparison of Pre-PRTLI and ‘during’ PRTLI Random selection of 200 researchers from HEA database by CWTS Global/broad discipline analysis Peer Review Output of Centres / Programmes as above Humanities, Social Sciences Centres / Programmes 2-7 peers : range of papers International context Research

22. Research & Teaching/Learning Indicators • Research • Equipment, specialised facilities etc. • Scale of Research Activities /Output • Teaching & Learning (UG & PG) • Activity/processes • Content (Courses, format etc) • Collaboration • Joint Initiatives

23. Strategy and Policy Review • Steering Committee: • Meetings with Presidents/Directors of HEIs (Successful and un-successful PRTLI applicants) – on site visits • Meetings with a range of government departments/public bodies • Meetings with leading industry figures and representatives of industry • Submissions from a range of organisations • Strategy/management Indicators and benchmarking with pre-initiation of PRTLI; international comparisons.

24. Conclusions • ………….