Ranking Tertiary Education Systems

1. Ranking Tertiary Education Systems JamilSalmi Astana 15 June 2009

2. Benchmarking Tertiary Education Systems JamilSalmi Astana 15 June 2009

4. the power of rankings • public debate • Malaysia • France • Brazil

5. institutional behavior • positive reactions • increased data-based decision making • improvements in teaching and learning practices • mergers or institutional collaboration

6. institutional behavior (II) • negative reactions • less interest for low-income and minority students • distorted resource allocation to favor research over teaching • neglect of established institutional research practices • managing to the rankings (cheating?)

7. outline of the presentation • world-class university or world-class tertiary education system? • rankings of university systems • benchmarking tertiary education systems

8. the search for excellence my university is… more world-class than yours

9. the WCU disease • governments • institutions • strategic efforts • lobbying for resource concentration

11. the WCU disease • governments • institutions • World Bank

13. how do you recognize a world-class university? • everyone wants one • no one knows what it is • no one knows how to get one Philip G. Altbach

15. reality check • well-performing countries without world-class universities • and vice-versa

16. top 50 universities

17. comparison of rankings

18. reality check • well-performing countries without world-class universities • time dimension of alignment? • emerging economies vs. mature economies • country size • Harvard vs. Canada

19. reality check • institutional differentiation • different types of institutions for meeting various learning and training needs • regional engagement • technology transfer through human capital formation

21. evolution of Nokia income

22. outline of the presentation • world-class university or world-class tertiary education system? • rankings of university systems

23. SJTU ranking (2008)

24. adjusting for size

25. University World News

26. SJTU

27. QS SAFENational System Strength Ranking • US • UK • Australia • Germany • Canada • Japan • France • Netherlands • South Korea • Sweden • number & ranking • access to top U • flagship U • investment

29. Lisbon Council • ranking of university systems (17 OECD countries) • beyond research • multi-dimensional (access and completion, labor market outcomes, lifelong learning, responsiveness, ability to attract foreign students) • pioneer work

30. limitations • face same methodological questions as university rankings • focus on universities • research bias • composite index with arbitrary weights • statistical robustness • choice of indicators

31. outline of the presentation • world-class university or world-class tertiary education system? • rankings of university systems • benchmarking tertiary education systems

32. cross-country comparisons help increase knowledge is this flower big or small? it depends on the size of neighboring flowers

33. multi-dimension comparisons enriches the diagnosis how does this flower compare with the others? wider but shorter

34. purpose • improving performance • through comparisons • competitors • good practices • diagnosis (identification of areas for improvement) • definition of specific corrective interventions

35. purpose (II) • need to understand the determinants of performance • no consensus on what countries should do to improve their performance • wide variations in system performance with similar funding levels and common country characteristics

36. Brazil and Chile • public spending on tertiary education = .8% and .3% of GDP respectively • enrollment rates are 24% and 38% respectively

37. why is a comprehensive benchmarking tool important? • performance is more than • building world-class universities • enrolling students (equity and quality agenda) • learning & labor market outcomes linked to the totality of the education experience

38. approach • elaborating a theoretical framework • selecting indicators • finding / generating the data • analysis • diagnosis (areas for improvement) • identification of possible solutions

39. preliminary step • defining a system • US • EU • small states

40. elaborating the theoretical framework • distinction between performance and health of system • how good are the system’s actual outcomes? • does it operate under conditions known to lead to high performance? • definition of outcomes / outputs / results • identification of determinants and causality relationships • informed by empirical evidence

41. results framework • research production • technology transfer • quality and relevance of education and training • labor market outcomes • measuring learning outcomes

42. results framework • equity • values and behaviors • social and cultural engagement

43. determinants (inputs and processes) • access and equity (enrollment, institutional diversification, pathways) • quality and relevance (standards, teaching and learning, research, local engagement)

44. determinants (inputs and processes) • sustainable financing (resource mobilization, allocation, efficient use) • capacity to improve (system and institutional governance)

45. indicators and data • selecting and defining the right indicators • finding comparable and reliable data (statistical sources and surveys) • objective • verifiable

46. the end

47. Upgrade your knowledge – measure, assess and compare your universities! Benchmarking Tertiary Education

48. conclusions • exploratory work • different type of analysis • who? governments and donors • what: system lens vs. institutional • multi-dimensional

49. main challenges ahead • defining and measuring multiplicity of outputs • finding reliable data • linking results and causes to be able to take action