1 / 24

Mapping Diversity – The U-Multirank Approach to Rankings Gero Federkeil

Mapping Diversity – The U-Multirank Approach to Rankings Gero Federkeil. Workshop Universidade Nova de Lisboa , 29th June 2012. Presentation. Portuguese Universities in Global Rankings. A (Short) Critique of Existing Global Rankings. An Alternative Approach: U-Multirank.

eilis
Download Presentation

Mapping Diversity – The U-Multirank Approach to Rankings Gero Federkeil

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mapping Diversity –The U-Multirank Approach to RankingsGero Federkeil WorkshopUniversidade Nova de Lisboa, 29th June 2012

  2. Presentation Portuguese Universities in Global Rankings A (Short) CritiqueofExisting Global Rankings An Alternative Approach: U-Multirank Universidade Nova de Lisboa, 2012/06/29

  3. Portuguese Universities in Global Rankings Portugueseuniversitiesare not well representedin international rankings Hencethoserankingsare not a goodinstrumenttolookatthePortuguese HE system Universidade Nova de Lisboa, 2012/06/29

  4. Presentation Portuguese Universities in Global Rankings A (Short) CritiqueofExisting Global Rankings An Alternative Approach: U-Multirank Universidade Nova de Lisboa, 2012/06/29

  5. The coverage of Global Rankings Existing global rankings cover only a smallminorityof all universities EUA: Global University Rankings andtheir Impact. Bruxelles, 2011. Universidade Nova de Lisboa, 2012/06/29

  6. The Indicators Existing global rankingsarerankingsofinternationallyorientedresearchuniversitiesonly Universidade Nova de Lisboa, 2012/06/29

  7. The methodology: Ranking orthodoxy • Most users are interested in information about “their” field” • Institutional rankings give misleading averages across fields/units Ranking of whole institutions • Composite indicators blur profiles and strengths & weaknesses • There are neither theoretical nor empirical arguments for assigning specific pre-defined weights to single indicators Composite overall score • Small differences in the scores of indicators lead to big differences in league table positions • Give false impression of exactness (“Number 123 is better than number 127”) League table approach Universidade Nova de Lisboa, 2012/06/29

  8. Conclusions Existing global rankings cover only a smallminorityofuniversities Due totheirindicatorstheyonlymeasureresearchand cover internationallyorientedresearchuniversitiesonly Theydevaluateotherprofilesandmissions (teaching, LLL, regional engagement) Theyare a threattodiversity in highereducation Universidade Nova de Lisboa, 2012/06/29

  9. Presentation Portuguese Universities in Global Rankings A CritiqueofExisting Global Rankings An Alternative Approach: U-Multirank Universidade Nova de Lisboa, 2012/06/29

  10. The Project • Commissioned by the European Commission • 2-year feasibility project, 2009 – June 2011 • Report now available: • http://ec.europa.eu/education/higher-education/doc/multirank_en.pdf • JánFigel, the former European Commissioner for Education, Training, Culture and Youth: “- to allow stakeholders to make informed choices; - to help institutions to position themselves and improve their performance” • Two phases: • Design of new instrument • Testing the feasibility of new instrument Universidade Nova de Lisboa, 2012/06/29

  11. Specification of the Project • Five dimensions: • Teaching & learning • Research • Knowledge transfer • International orientation • Regional engagement • Development of a list of indicators to be tested in pilot project • Development of data collection tools and processes (question-naires, definitions, FAQs, communication + feedback processes) • Done by CHERPA consortium: CHE, CHEPS, CWTS, INCENTIM; OST Universidade Nova de Lisboa, 2012/06/29

  12. Specification of the project • Two levels: • Institution (FIR) • Fields (FBR) • Global sample of higher education and research institutions: • 159 (target: 150), 2/3 Europe, • 109 completed institutional questionnaires • Two pilot fields: • Business studies • Engineering (electrical and mechanical) Universidade Nova de Lisboa, 2012/06/29

  13. + + + The Basic Methodology Multi-dimensional ranking • There is no single objective ranking • Each ranking reflects the ideas and preferences of those doing them • The decision about the relevance of indicators should be left to the user Multi-level ranking • Different levels of analysis are relevant for different users • Field-based and institutional rankings Grouping approach • Rank groups instead of league tables provide more meaningful and valid information Universidade Nova de Lisboa, 2012/06/29

  14. Mapping Diversity • A basic aim of U-Multirank is to make visible the diversity of Higher Education Institutions • and to show excellence beyond research excellence HOW ? Universidade Nova de Lisboa, 2012/06/29

  15. Startingpoint: Diversityofhighereducationinstitutions in Europe & theworld Identifyingcomparableinstitutionsthatcanbecompared in oneranking Classification Rankings Description of horizontal diversity  Types/profiles Assessmentofvertical diversity  Performance + Complementaryinstrumentsoftransparency Universidade Nova de Lisboa, 2012/06/29

  16. Mapping and Ranking • Step: Mapping: • Selection of a comparable set of universities based on institutional profiles Teaching andlearning • Example: • Comprehensive, teachingorientedinstitution • Mainlyundergraduateeducation • Low researchorientation • Low international orientation • Regionallyembedded (e.g. recruiting) Research involvement Knowledgeexchange International orientation Regional engagement 2. Step: Ranking of subsetofcomparableinstitutionswithsimilarprofiles Student profile Universidade Nova de Lisboa, 2012/06/29

  17. Multi-Dimensional Ranking for Subset of Comparable Institutions Nocompositeindicator! Nonumber 1 ! Universidade Nova de Lisboa, 2012/06/29

  18. User-driven, Personalised Ranking Universidade Nova de Lisboa, 2012/06/29

  19. A Personalised Ranking … … Helpingtomake an informedchoice based on invidualpreferences Universidade Nova de Lisboa, 2012/06/29

  20. Results of the feasibility study • Generally theconcept, indicatorsandinstrumentsof U-Multirank arefeasibleboth on theinstitutionalandthefieldlevel • Therearesomeproblemsconcerningindicators, mainly on • Issuesofemployability • Knowledgetransfer, and, • Regional engagment • Therehastobesomerefinementofconceptsandindicators Universidade Nova de Lisboa, 2012/06/29

  21. Outlook : U-Multirank II • New callfortenderlaunchedbythe European Commission • CHE appliedwithpartners (CHEPS, CWTS, …) • Decisionatthe end ofJuly (?) • Start oftheproject on 1 October (?) • 2 * 2 years • Implementationoftheconcept: First publicationofrankingatthe end of 2013 • Min. 500 institutions, institutionaland 4 fields • Annual extensionofnumberofinstitutionsandfields • Development of a business model for a sustainablesystem Universidade Nova de Lisboa, 2012/06/29

  22. „You‘re kidding! You count publications?“ But after all, there still might be some limits to ranking…

  23. Muitoobrigado! Formoreinformation: www.u-multirank.eu gero.federkeil@che.de

  24. Mapping Diversity –The U-Multirank Approach to RankingsGero Federkeil, CHE Centrefor Higher Education WorkshopUniversidade Nova de Lisboa, 29th June 2012

More Related