What Can National Rankings Learn from the U-Multirank-Project ?

1. What Can National Rankings Learnfromthe U-Multirank-Project ? Gero Federkeil, CHE, Germany IREG-Forum: National University Rankings on the Rise Bratislava, 10-11 Oct. 2011

2. Part 1 - U-Multirank basics IREG Forum | Gero Federkeil | Bratislava 2011-10-10

3. The project • Commissioned by the European Commission • 2-year project, 2009 – June 2011 • Report now available: http://ec.europa.eu/education/higher-education/doc/multirank_en.pdf • JánFigel, the former European Commissioner for Education, Training, Culture and Youth: “- to allow stakeholders to make informed choices; - to help institutions to position themselves and improve their performance” • Two phases: • Design of new instrument • Testing the feasibility of new instrument IREG Forum | Gero Federkeil | Bratislava 2011-10-10

4. Specification of U-Multirank • Five dimensions: • Teaching & learning • Research • Knowledge transfer • International orientation • Regional engagement • Long list of indicators to be tested in pilot project • development of data collection tools and processes (question-naires, definitions, FAQs, communication + feedback processes) • methods for building ranking groups instead of league tables IREG Forum | Gero Federkeil | Bratislava 2011-10-10

5. Testing U-Multirank • Two levels: • Institution (FIR) • Fields (FBR) • Global sample of higher education and research institutions: • 159 (target: 150), 2/3 Europe, • 109 completed institutional questionnaires • Two fields: • Business studies • Engineering (electrical and mechanical) IREG Forum | Gero Federkeil | Bratislava 2011-10-10

6. Baislogic: Mapping Diversity Diversityofhighereducationinstitutions in Europe & theworld Identifyingcomparableinstitutionsthatcanbecompared in oneranking Classification Rankings Description of horizontal diversity  Types/profiles Assessmentofvertical diversity  Performance + Complementaryinstrumentsoftransparency IREG Forum | Gero Federkeil | Bratislava 2011-10-10

7. Mapping and Ranking Mapping: Selection of a comparable set of universities based on institutional profiles Teaching andlearning Example: • Comprehensive, teachingorientedinstitution • Mainlyundergraduateeducation • Low researchorientation • Low international orientation • Regionalylembedded (e.g. recruiting) Research involvement Knowledgeexchange International orientation Regional engagement Subsetofcomparableinstitutionstobecompared in a ranking Student profile IREG Forum | Gero Federkeil | Bratislava 2011-10-10

8. Mapping and Ranking Ranking: Multi-dimensional ranking for subset of institutions Nocompositeindicator! Nonumber 1 ! IREG Forum | Gero Federkeil | Bratislava 2011-10-10

9. Part 2 (What) can national rankings learn from U-Multirank? IREG Forum | Gero Federkeil | Bratislava 2011-10-10

10. Mapping and Ranking Most national HE systemsarediversified HE systems: Different types/profilesofinstitutionsexist  Need toidentifycomparableinstitutionsforranking  Mapping systemscanincreasethecomparabiliyandimprovethequalityofrankings  U-Mapdefinesindicatorsformapping & issetting a standardfor Europe IREG Forum | Gero Federkeil | Bratislava 2011-10-10

11. Multi-dimensional Approach • Multi-Multirank identified a setofindicatorsfor 5 dimensions • U-Multirank introduced 2 „new“ dimensions: • knowledgetransfer • regional engagement • Indicatorshavebeendiscussedintensivelywithstakholders IREG Forum | Gero Federkeil | Bratislava 2011-10-10

12. Innovative indicators Teaching and learning: For rankings which want to inform (prospective) students indicators based on students‘ assessment of their teaching and learning experience are highly useful and are feasible (in most settings) • Knowledgetransfer: • Joint publicationswithindustry • Research fundsfromindustry • But problemswithregardtodata (e.g. on spinoffs/licenes) IREG Forum | Gero Federkeil | Bratislava 2011-10-10

13. Innovative indicators International Orientation Rating indicator on international orientationof programmes ismoremeaningfulthanlinear rankingofnumberof int. students • Regional Engagement: • Importantformany HEIs yetmostproblematicdimension in U-Multirank • Bibliometric indicator: Regional co-publications • Further developmentisnecessary IREG Forum | Gero Federkeil | Bratislava 2011-10-10

14. User-driven Approach • Intensive stakeholderconsultationhelpedtoincreaseacceptance • Multi-dimensional, personalisedrankingsallow • individual userstoproducerankingbased on theirownpreferencesand • networksandaossciationsofuniversitiestostartbenchmarking / createtheirownranking IREG Forum | Gero Federkeil | Bratislava 2011-10-10

15. Data collection • International rankings have to rely on self-reported data due to lack of international data bases (except bibliometric, patent data) • Feedback loop with universities concerning self-reported data on institution, faculties & programmes helped to increase consistency & quality of data • Parallel / conflicting national data collections (e.g. student surveys) •  raises issue of coordination national – international rankings in general IREG Forum | Gero Federkeil | Bratislava 2011-10-10

16. Outlook IREG Forum | Gero Federkeil | Bratislava 2011-10-10

17. Outlook: National rankingsand U-Multirank • There will be a continuingdemandfornationalrankings ! • Definition of a coresetofindicatorsfor national rankingsand U-Multirank? • Network of national rankings, e.g. Germany – Austria – Switzerland - Netherlands – Spain …. thatsharedatawhichcanbeusedfor U-Multirank IREG Forum | Gero Federkeil | Bratislava 2011-10-10

18. Thankyouverymuch! More information: Gero.federkeil@che-ranking.de www.che-ranking.de www.u-multirank.eu

19. What Can National Rankings Learnfromthe U-Multirank-Project ? Gero Federkeil, CHE, Germany IREG-Forum: National University Rankings on the Rise Bratislava, 10-11 Oct. 2011