Times Higher - QS World University Rankings Methodology & Europe’s Performance

1. Times Higher - QS World University RankingsMethodology & Europe’s Performance Ben SowterHead of ResearchQS University of Edinburgh10 March 2008

2. Speaker Introduction • Graduated in Computer Science from University of Nottingham in 2000 • Union Prize • Chair, Nottingham University Debating Society • Spent 3 months on work placement in northern India in 1999 • Worked for international student charity – AIESEC – for two years after graduation • 2nd Year as UK National President • Operated on international strategic development team • Joined QS in 2002

3. Objectives • Establish QS background & credentials • Explain research methodology • New developments • Understand limitations of rankings • Provide insight into performance enhancement specific to improved ranking • Collect feedback and input

4. Introduction • QS • Why World University Rankings? • THE – QS Collaboration • Our Approach • Results • UK & Ireland in the Rankings • Progress in the Rankings

5. QS A brief introduction

6. Founded 1989/1990 • Nunzio Quacquarelli • MBA Wharton • The MBA Career Guide (now QS Top MBA Career Guide) • MBA Events • Grad School/Postgraduate Events • Careers • Software • Scholarships

7. QS Mission… …to enable motivated people around the world to fulfill their potential, by fostering international mobility, educational achievement and career development Our Channels • primary research • leading-edge editorial & publications • developmental events • web solutions

8. www.topuniversities.com

9. www.topuniversities.com • Over 1,180,000 visits in 2007 • Over 6,000 visits per day in 2008 • Increasingly strong prominence in Google & Yahoo searches • Ranked 53,626 in Alexa • Home of the THE– QS World University Rankings • Institution Profiles • Detailed study abroad information

10. Top Universities Guide • New revised and updated edition available now • Contains much information not available on web

11. Other QS Initiatives • QS World Grad School Tour • QS Top Grad School Guide • QS World MBA Tour • QS Top MBA Career Guide • QS APPLE – Korea July 2008 • QS Top Apply • QS Education Trust • QS Intelligence Unit

12. QS Offices & Events

13. Why rank universities? • Interest in ranking things and people • Hospitals • Schools • Local authorities • Rich lists; Britain, world, Asian British, footballers • Universities: The Times / US News etc…

14. Why World Rankings? • Higher education becoming more global • Knowledge the key driver of international competitiveness • Increasing desire for comparative information • Unique positions of THE and QS as international and independent experts in higher education • Raises awareness of all 500+ universities involved in the project

15. International Study Trends • Worldwide, the demand for international education is forecast to increase rapidly • In 2003, an estimated 3.1 million students studied internationally • There are likely to be in excess of 5.8 million international students in 2010 • Two-thirds of this global demand will be Asians • US domination of international student recruitment has begun to falter – tight Visas, increased competition (IIE, NAFSA, CCE) • Bologna Accord will create an ‘explosion in English-language based postgraduate courses in Europe’

16. Student Mobility on the Rise

17. International Students Abroad Source: IIE – Institute of International Education

18. International Study Trends • Government funding of tertiary education being cut (per capita) in most countries around the world (IFC) • Many governments targeting international students as a source of tertiary sector funding • Australia has estimated that international students are more important to their economy than manufacturing & mining • The UK Government estimates that 270,000 students contribute $3 billion in fees & a further $3 billion in other spending

19. International Study Trends • By 2010 The European Commissions Bologna Accord will make room for over 500,000 first degree graduates to study in other EU nations for a Masters degree • By 2016 In the USA, recent Abraham Lincoln Study Abroad legislation aims to encourage 1 million students to study overseas • By 2020 Over 3 million Asians are expected to study outside their home country (Source: IDP – British Council)

20. World University Rankings can help differentiate the active, from the dormant…volcanoes! Professor Shih, President National University of Singapore

21. Academic Ranking of World Universities • Operated by Shanghai Jiao Tong University in China • Focuses principally on research output • Nobel Prizes & Fields Medals • Citations • HiCis • Strongly correlated with educational reputation • Historical dependency • Available on www.arwu.org

22. Shanghai Jiao Tong Criteria

23. Shanghai Jiao Tong – Issues? Strengths? Weaknesses? Nobel prizes look backwards Citation data is historic Citation measures bias towards sciences and bio-sciences Overall bias towards US universities • All data is easily verifiable • First global ranking established • Very mechanical methodology

24. Webometrics Ranking of World Universities • Focuses on web profile • Research productivity using Google Scholar • Search Engine performance • Site popularity • Good practices in web management • Fully automated • Tracks in the realm of 14,000 institutions • Prone to anomaly • A university domain change can be disastrous • Available on www.webometrics.info

25. Webometrics Criteria

26. World University Rankings

27. THE - QS World University Rankings • Began in 2004 • Collaboration between Times Higher Education (THE) & QS • THE running since 1971 • THE is a weekly newspaper distributed to academics in the UK and internationally • Formerly associated with The Times • Published annually in the fall

28. THE – QS World University Rankings THE METHODOLOGY

29. Our Approach • Four key themes, as basis for comparison: • Research Quality • Graduate Employability • International Outlook • Teaching Quality

30. Ranking Criteria & Weights *All decisions regarding the allocation of weightings are the responsibility of the Times Higher Education Supplement

31. Selection of Initial List • Began in 2004 with list of the top 500 world institutions by research impact • Excludes single-faculty institutions • Excludes postgraduate-only institutions • Added on recommendation from certain countries where English publication culture is not strong (e.g. Germany) • Each year list is re-evaluated • Omitted institutions are welcome to make a case for inclusion with supporting evidence for their inclusion relative to an included institution • 566 universities considered in 2007

32. Peer review is an effective way to evaluate universities. It takes smart people to recognise smart people Sir Richard Sykes Rector, Imperial College

33. Peer Review of Research Output • Peer review is the centrepiece of this ranking • 5,101 respondents • International spread • Weighted Results • Subject spread • Arts & Humanities • Engineering & IT • Life Sciences & Biomedicine • Natural Sciences • Social Sciences • Overall score built up from each faculty area • Active academics • 3 year “latest response”

34. Peer Review – Selection of Peers • 3 year latest response • Invitation to previous respondents • World Scientific Database (180,000) • Singapore Headquarters • www.worldscientific.com • Mardev Database (12,000+) • Part of Reed Elsevier • UK Headquarters • www.mardev.com • Academic Sign-up facility • www.topuniversities.com

35. Observations: Peer Review • Harvard second to Berkeley • Oxford and Cambridge • Well-liked universities all over the world • Little evidence of patriotism bias • UK and North America dominate the top 20

36. The Recruiter Review

37. Recruiter Review • Another group who have knowledge of university quality • Key relevance to “graduate employability” criteria • Introduced for 2005 rankings • Sourced from QS database and media and university referrals (over 120 lists received for 2007) • Improved response for 2006 & 2007 • Continuing development focus for 2008

38. Observations: Recruiter Review • UK • Strong but less so than in previous years • Some institutions with particular strengths do well • Bocconi, HEC

39. Quantitative Measures

40. Quantitative Measures • Research Quality • citations/faculty • Teaching Quality • faculty student ratio • International Outlook • % international faculty and students • Graduate Employability • ??

41. Observations: Staff/Student • Winner – Caltech • US, French, Swiss, Netherlands etc institutions all well placed • Both Asian and European amongst the well-placed • Weak correlation (0.21) with research – but not zero

42. Observations: International • US shows up badly • HKUST top in staff • London School of Economics top in students • Caltech among few US with international staff

43. Citations per Faculty Score • Classic measure of research quality • Citations per staff member (not per paper) • 5 year score (changed from 10 in 2006 to increase “topicality”) • Source: Scopus

44. Observation: Citations • Medical faculty is a big plus • Or major biomedical income • CalTech the winner, then Stanford& MIT • Big country effect

45. Z-Score Aggregation • Method used in various domestic rankings such as The Times (UK) • Stabilise data year on year • More accurate application of weightings throughout the sample • Smoothes any anomalies that remain • Essentially smoothes the curve for each individual indicator

46. The Effect of Z-Score Application on Rankings Indicators

47. 2007 Indicator Correlations

48. Effects of 2007 developments… • Online System • More thorough and complete data all the way down the list • Less need for averages • Peer Review • Countries with disproportionately high response but a small number of institutions lose unnatural advantage • FTE Data • Universities with particularly high or low numbers for part-time will be influenced • Scopus • Less pronounced bias towards US and English speaking world • Z-Score aggregation • Less advantage for the best institutions in lower weighted indicators • Increased stability year on year

49. NOTE • In understanding and using any ranking, it essential to develop a full understanding of what they are measuring • THE & QS are committed to: • Being open with our methods and data wherever possible • Engaging with universities and other stakeholders to help them understand the rankings • Providing as much qualitative information in support of the rankings as possible

50. World University Rankings - Issues? • The ranking has limitations. Lack of data on: • Teaching quality • Student satisfaction • Investment in infrastructure • The ranking relies on the best comparable data available