330 likes | 336 Views
Explore a new ranking method that evaluates educational institutions on multiple dimensions, avoiding biases and promoting cooperation and communication.
E N D
U-Multirank On the feasibility of a new approach Frans Kaiser, CHEPS Mini-seminar organised by the Transparency Tools Working Group for the BFUG Cracow 12 October 2011
Rankings in highereducation • instruments to compare higher education institutions • and to judge their relative position, usually in league tables • based on the actual performance of these institutions
The rise of global rankings • Academic Ranking of World Class Universities (ARWU) Shanghai Jiaotong University, since 2003 • Times Higher Education Supplement World Rankings (THE) Times Higher Education, since 2004 • Higher Education Evaluation and Accreditation Council of Taiwan Ranking (HEEACT), since 2007 • The Leiden Ranking (LR) Leiden University, since 2008
Critique of existing rankings • focus on ‘whole institutions’ (ignoring internal variance) • concentrate on ‘traditional’ research productivity and impact • focus on ‘comprehensive research universities’ • aggregate performance into composite overall indicators • use constructed ‘league tables’ • imply cultural and language biases • imply bias against humanities and social sciences
Designing an alternative: the EC call for Tender • development of concept and feasibility study of a global ranking (not only European) • multi-dimensional, not only research • multi-level (institutional and ‘field’) • all types of higher education and research institutions • multiple stakeholders
Newapproach to ranking Subset of institutional activity profiles
Newapproach to ranking Subset of institutional activity profiles
Institutional performance profiles multidimensional perspective of ‘institutional profiles’ no overall ‘league tables’ no composite institutional indicators two-level analysis (institutional and ‘field’) stakeholders driven approach & specific ‘authoritative rankings’ allowing ‘performance profiles’ in various dimensions base for benchmarking, for inter-institutional cooperation, for effective communication and profiling base for informed governmental differentiation policies at system level
Pilot • Two levels: • Institution (FIR) • Field (FBR) • Global sample of higher education and research institutions: 159 (target: 150), 2/3 Europe, 109 completed institutional questionnaires • Two fields: • Business studies • Engineering (electrical and mechanical)
Pilot • Self-reported institutional data by means of four online questionnaires: • U-Map (Institutional) • U-Multirank (Institutional level) • U-Multirank (department level) • Student survey • International databases • Bibliography • Patent data
Pilot: conclusion Thereareissuestoberesolved (like regional engagement, employability) but itmaywork Whatcoulditlooklike?
Choose the type of ranking you are interested in. Ranking of Business studies programs Ranking of institutions Ranking of Electrical engineering programs
Selection of matching institutions (1) Doctorate production Other research products Academic publications Distance education Total enrolment % Expenditure on research Cultural activities Start-up firms Orientation of degrees % Expenditure on teaching Income knowledge transfer Subjects covered Mature students Teaching & learning Student profile Research involvement Knowledge transfer Degree level focus Part time students Patent applications Professional publications
Selection of matching institutions (2) Exchange stud; incoming Non-national academic staff Foreign degree seeking stud Graduates in the region Income from regional sources Income from internat sources International orientation Regional engagement Exchange stud; sent out New entrants from region Continue to ranking
Performance profiles (institutional level) Student internships in region Graduationratebac % incomefrom the region Graduationrate master Time todegreebac Research contracts withregionalenterprise Regional joint research publications Time todegree master % exp on teaching Graduatesworking in the region Graduateunemployment % interdisciplinary programs Highlycited research publications Patents awarded Field normalisedcitationrate Post docs per academicstaff Start up firms Art related research output Size of TTO % of research incomefromcompetitive sources Co-patenting % incomethird party funding Createpersonalised ranking table % expenditure on research Incentives forknowledge transfer Interdisciplinary research Univ-industry joint publ. CPD courses offered International doctorategraduationrate. International joint research publications Research publication output. % programs in foreignlanguage. % internationalstaff % students in joint degree programs 293 148 152 4 222 111 98 196
Personalise your ranking Teaching & learning Research Knowledge transfer International orientation Regional engagement Show personalisedtable
Personalised institutional ranking sort 293 148 152 4 222 111 98 196
Personalised institutional ranking Sorted by indicator #1 sort 293 148 152 4 222 111 98 196
Personalised institutional ranking Sorted by indicator #3 sort 293 148 152 4 222 111 98 196
Background information Name of institution: 293 Address URL U-Multirank profile Mission statement U-Map profile
Choose the type of ranking you are interested in. Ranking of Business studies programs Ranking of institutions Ranking of Electrical engineering programs
Selection of matching institutions (1) Doctorate production Other research products Academic publications Distance education Total enrolment % Expenditure on research Cultural activities Start-up firms Orientation of degrees % Expenditure on teaching Income knowledge transfer Subjects covered Mature students Teaching & learning Student profile Research involvement Knowledge transfer Degree level focus Part time students Patent applications Professional publications
Selection of matching institutions (2) Exchange stud; incoming Non-national academic staff Foreign degree seeking stud Graduates in the region Income from regional sources Income from internat sources International orientation Regional engagement Exchange stud; sent out New entrants from region Continue to ranking
Default business studies ranking Default field based ranking 293 148 152 4 222 111 98 196
Default business studies ranking Default field based ranking Sorted by indicator #1 293 148 152 4 222 111 98 196
Default business studies ranking Default field based ranking Sorted by indicator #12 personalise 293 148 152 4 222 111 98 196
Personalise your ranking Teaching & learning Teaching & learning; student satisfaction Research International orientation Knowledge transfer Regional engagement Show personalisedtable
4 Personalised ranking business studies 98 115 138 139 144
Personalised ranking business studies Sorted by indicator #1 251 115 292 546 4
Personalised ranking business studies Sorted by indicator #3 613 4 546 144
Discussion No ‘U-Multirank league table’: lesspresscoverage, betteruse? Authorativerankings: rankings in a user-defined setting? U-Multirank; a new approach? ** Multi-dimensional * Multi-level *** Inclusive ** Comparability check (U-Map) **** User-driveness
Thankyouforyour attention www.u-multirank.eu