THE WORLD UNIVERSITY RANKINGS History of the Rankings and the Global higher education context

1. THE WORLD UNIVERSITY RANKINGS History of the Rankings and the Global higher education context Phil Baty Editor Times Higher Education World University Rankings

2. About Times Higher EducationThe weekly magazine for all higher education professionals

3. Why Rank? Rapid globalisation of higher education • There are 3.7 million students enrolled in higher education outside their country of origin – 7 million by 2020 • Universities now have at least 200 satellite campuses outside their home countries (37 more on their way) • Around 20 per cent of all academics working in the UK are appointed from overseas • Almost 40 per cent of papers from top 200 universities international • Sir Drummond Bone said: “World class research is inherently international”

4. Why Rank? Rapid globalisation of higher education • “We are living through one of those tipping points where in five years, (commentators will say) that this was the period when the landscape changed for ever, when the speed of reputational growth and decline suddenly accelerated. • “We all accept that higher education is borderless - ideas know no boundaries, do not accord any significance to geography and maps - and that is equally true of reputations and university rankings.” • Peter Upton • Director, British Council, Hong Kong

5. Why Rank? Rankings have a useful function • “Rankings often serve in place of formal accreditation systems in countries where such accountability measures do not exist.” • “Prompt change in areas that directly improve student learning experiences” • “Encourage institutions to move beyond their internal conversations to participate in broader national and international discussions.” • “Foster collaboration, such as research partnerships, student and faculty exchange programmes.” • US Institute for Higher Education Policy, May 2009

6. Rankings: increasing influence “Rankings are an unmistakable reflection of global academic competition… they seem destined to be a fixture on the global education scene for years to come… As they are refined and improved they can and should play an important role in helping universities get better.” Ben Wildavsky, The Great Brain Race (Princeton University Press, May 2010)

7. The old ranking system: 2004-2009, with QS Citations Reputation Intl students Intl staff Staff-Student ratio Employer poll

8. Old QS ranking system (2004-2009) not fit for purpose We have torn them up and will start again QS no longer has ANY involvement at all with the Times Higher Education World University Rankings We abandoned the old THE-QS methodology and developed a new system in consultation with academics and university managers worldwide.

9. Old QS ranking system (2004-2009) not fit for purpose “Results have been highly volatile. There have been many sharp rises and falls… Fudan in China has oscillated between 72 and 195…” Simon Marginson, University of Melbourne. “Most people think that the main problem with the rankings is the opaque way it constructs its sample for its reputational rankings”. Alex Usher, vice president of Educational Policy Institute, US. “The logic behind the selection of the indicators appears obscure”. Christopher Hood, Oxford University

10. Old QS ranking system (2004-2009) not fit for purpose “The organizations who promote such ideas should be unhappy themselves, and so should any supine universities who endorse results they view as untruthful” Andrew Oswald, professor of Economics, University of Warwick, 2007.

11. Times Higher Education’s responsibility “The responsibility weighs heavily on our shoulders. We are very much aware that national policies and multimillion-pound decisions are influenced by the rankings…. We feel we have a duty to improve how we compile the rankings.. “We believe universities deserve a rigorous, robust and transparent set of rankings – a serious tool for the sector, not just an annual curiosity.” Ann Mroz, Editor, Times Higher Education, November 2009

12. What was wrong with the old QS system? Our editorial board highlighted two key concerns: * Reputation survey * Citations

13. What was wrong with the old QS system? Reputation Peer review – simply a reputation survey. Inherently controversial Subjective. They reflect past, not current, performance. Based on stereotype or even ignorance. A good or bad reputation may be mindlessly replicated. But: support for reputation measure in Thomson Reuters’ opinion poll. 79 per cent said were a “must have” or “nice to have”. Reputation is crucial. Survey can bring in some measure of the things quantitative data cannot.

14. What was wrong with the old QS system? Reputation QS achieved a tiny response rate to its survey: In 2009 only around 3,500 people responded to the survey Tiny number of responses from individual countries. In 2008, there were just 182 from Germany, and 236 from India. Lack of clarity over the questions asked. What are we judging? This is not good enough when you’re basing 40 per cent of the score on academic peer review

15. What was wrong with the old QS system? Reputation “The scores are based on a rather small number of responses: 9,386 in 2009 and 6,534 in 2008: in actual fact, the 3,000 or so answers from 2009 were simply added to those of 2008. the number of answers is pitifully small compared to the 180,000 e-mail addresses used”. Global University Rankings and Their Impact, European Universities Association, June 2011

16. What was wrong with the old QS system? Reputation “What are the criteria for leaving out a great number of universities or whole countries? The lists of universities pre-selected.. Usually continued universities from only 25/26 European countries out of the 48 countries in the European Higher Education Area”. Global University Rankings and Their Impact, European Universities Association, June 2011

17. What was wrong with the old QS system? Reputation “QS’s extensive corporate database, a network of partners with whom QS cooperates in its events, and participating institutions who submit a list of professionals with whom they work, thus creating a new bias”. Global University Rankings and Their Impact, European Universities Association, June 2011

18. What was wrong with the old QS system? Reputation “The fact that the total number of worldwide responses in 2009 was only 2,336 has implications. .. Such a small sample of world employers might simply not be aware of excellent universities in smaller countries, and especially in those countries where neither English nor another world language are spoken”. Global University Rankings and Their Impact, European Universities Association, June 2011

19. What was wrong with the old QS system? Citations QS failed to take into account dramatically different citation volumes between disciplines Major bias towards hard sciences, because arts and humanities papers have much lower citation volumes No normalisation for subject

20. What was wrong with the old QS system? Citations

21. What was wrong with the old QS system? Citations

22. Influence despite clear flaws “The term world class universities has begun to appear in higher education discussions, in institutional mission statements, and government education policy worldwide” “Many staffing and organisational decisions at institutions worldwide have been affected by ranking-related goals and outcomes.” “Rankings play an important role in persuading the Government and universities to rethink core national values” US Institute for Higher Education Policy

23. The development of a new world ranking system In November 2009 we signed a deal with Thomson Reuters, to work with us to develop and fuel a new and improved global ranking for the future.

24. A perfect partner • “In addition to unmatched data quality, Thomson Reuters provides a proven history of bibliometric expertise and analysis. We are proud that our data continues to be chosen by leading organisations around the world and we’re happy to provide insight and consultation on such a widely respected indicator,” • Jonathan Adams, director of research evaluation, Thomson Reuters

25. Thomson Reuters’ stakeholder survey. Key findings: “The overriding feeling was that a need existed to use more information, not only on research, but also on broader institutional characteristics. The data indicators and methodology currently utilized were perceived unfavorably by many and there was widespread concern about data quality in North America and Europe.” Global Opinion Survey: New Outlooks on Institutional Profiles

26. Thomson Reuters’ stakeholder survey. Key findings: “Some would even manipulate their data to move up in the rankings. This is of great concern and warns against any reliance on indicators that could be manipulated without creating a real underlying improvement...” Global Opinion Survey: New Outlooks on Institutional Profiles, Feb 2010

27. Thomson Reuters’ stakeholder survey. Key findings: • 92 per cent said that faculty output (publications) was a must have/nice to have • 91 per cent said that faculty impact (citations) was a must have/nice to have • 86 per cent said they wanted faculty/student ratios • 84 per cent wanted income from research grants • 79 per cent wanted peer “reputation” measure

28. Thank you • • Visit the Global Institutional Profiles Project website:http://science.thomsonreuters.com/globalprofilesproject • • See the results in full, with our interactive tables: http://bit.ly/thewur • • Join our rankings Facebook group. www.facebook.com/THEWorldUniRank • • Keep up to date with all the rankings news on Twitter: @THEWorldUniRank • * Follow Phil Baty on Twitter: @Phil_Baty

29. Thank you. Stay in touch.Phil BatyTimes Higher EducationT. 020 3194 3298E. phil.baty@tsleducation.com