Edouard Mathieu Head of the Benchmarking Center Invest in France Agency

1. A few remarks on ARWU* Edouard Mathieu Head of the Benchmarking Center Invest in France Agency * ARWU: Academic Ranking of World Universities 2005

2. Some remarks on methodological issues • ARWU is a useful idea. The difficulties are honestly documented by prof. Liu. Some will be hard to overcome. • Among difficulties: the attribution of citations (example in Paris: Paris-6 or UPMC, careers in CNRS laboratories, College de France, Polytechnique…). Does Shanghai Jiao Tong University benefit from a partner in France to help with the French attributions? • Nobel and Fields medals are skewed indicators: only 12 universities score > 50 for alumni ; 305 universities score 0 ; 372 universities score 0 at awarded staff. Beyond the top 10, these variables do not discriminate academic performances. • Only one measure for all world diverse universities and schools makes global ranking difficult to interpret. We should rather compare comparable units.

3. How to gain 10 ranks? Paris-6 versus UT at Austin • Ranking. Paris-6 (Université Pierre et Marie Curie, UPMC) ranks 46, UT Austin (The University of Texas at Austin) ranks 36. Why? • Enrollment. Paris-6 30,000 (science and medical students); UT Austin 50,000 (architecture, liberal arts, business, law, natural sciences, engineering, pharmacy, nursing…) • Employees. Paris-6 : 4,000 faculty and 3,000 staff; UT Austin : 3,000 faculty and 18.000 staff (laboratories, museum, library, sports…). UT Austin enjoys big budget and managing powers. • Scores. Paris-6 much better at alumni, awards, size (per capita) and equal at SCI. UT Austin better only at HiCi and N&S. • Conclusion. Paris-6 matches UT Austin at academic performances. Size not an issue here. Strong English language bias.

4. Remarks on communication issues • ARWU is meant to measure scientific competences, but it is generally perceived as a global ranking, including teaching performances and efficiency. • Academic indicators are included because they are related to the “reputation “. It implies that ARWU reinforces initial reputations ; a ranking should rather help detect good practices. • The relationships between academic performances, teaching performances and efficiency at meeting the economy needs should be somewhat elaborated. • Beyond the top ten, it would be much more appropriate to communicate only classes of equivalent level universities : UT Austin and Paris-6 would play in the same league (World class) beyond the World top twenty, and not ranked 36/46. ARWU already uses such classes beyond 100: classes 101-152, 301-400.

5. Impact on university management • Most indicators are absolute numbers, and so ARWU favors big size. Is bigger better than smaller? No universal answer. • Paris (as other French cities) boasts many specialized universities, schools and organizations. This dispersion may send the wrong message through ARWU. Some experiences are made to join forces for better visibility: Paris Tech… • Prof. Liu intends to increase the weight of size (per capita criteria). This will make ARWU more neutral on the size question. But comparable evaluations of faculty and staff numbers will not be easy to elaborate. • Same difficulties with the specialization / concentration management alternatives and corresponding rankings.

6. Impact on student international mobility • ARWU should be a helpful tool for mobility (both ways). Is it? • Probably a significant impact: 2000 visitors on the Web a day (Chinese? From developed countries?) • Probably less impact on the top excellent students who receive direct offers and information from universities and schools. • Impact on professors / researchers national and international mobility ? (which would boost future impact on student mobility).

7. Academic ranking and countries' attractiveness • Top 500 Universities list gives a ranking of countries for research : US (1st), UK, Japan, Germany, Canada, France(6th), Sweden, … China (19). How it compares with rankings in the “France attractiveness scoreboard”? • Proportion of international students enrolled in tertiary education: Australia, Belgium, UK, Germany, France, USA, Netherlands, Spain, Japan… • R&D expenditure/GDP (average 2001-2003): Japan, USA, Germany, France and Belgium (ex.), Netherlands, UK… • ARWU and the Attractiveness Scoreboard agree on the global picture. But France’s 6th global position is a more solid assertionthan separate universities rankings, with a 46th ranking for Paris-6, 101-152d ranking for College de France.

8. Edouard Mathieu Head of the Benchmarking Center Invest in France Agency