1 / 42

Mauro Degli Esposti

Towards a new model for evaluating the merit of Italian academic scientists and their institutions. Mauro Degli Esposti. Ferrara, Jan 2011. A new analysis of Italian universities – the start. From The Times October 9, 2008 Whatever happened to the University of Bologna?

hong
Download Presentation

Mauro Degli Esposti

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards a new model for evaluating the merit of Italian academic scientists and their institutions Mauro Degli Esposti Ferrara, Jan 2011

  2. A new analysis of Italian universities – the start From The Times October 9, 2008 • Whatever happened to the University of Bologna? • The University of Bologna, which received its charter from Frederick I Barbarossa in 1158, was probably the first university founded in the Western world. It became a renowned home to scholars of great distinction in both the sciences and the humanities. The Times Higher Education-QS World University Rankings survey was published this week. It rates the performance of universities according to the quality of their research, the employment rates of graduates and their international profile. • The University of Bologna does not feature in the top 100! Harvard = 1, Cambdrige = 3, Oxford = 4, UCL = 7... Manchester = 29 – now 26 M Degli Esposti, presented at the first conference of Via-academy, 21 September 2009 http://via-academy.org/Relazione_Prima_Conferenza_della_VIA_a_Manchester.aspx

  3. Diritto di cronaca 12/10/2009 Un ateneo per scappare di FLAVIA AMABILE MERIT – a new approach to evaluation The members of the Via-academy network who work in UK have been selected on the basis of merit, but educated in Italy. The number of academic links of Via-members with Italian institution will be proportional to the relative teaching capacity of these institutions – i.e. they will be an output measure of their quality and relative standing. M Degli Esposti, presented at the first conference of Via-academy, 21 September 2009 http://via-academy.org/Relazione_Prima_Conferenza_della_VIA_a_Manchester.aspx

  4. by Marco Geraci M Degli Esposti, presented at the first conference of Via-academy, 21 September 2009

  5. VIA25 - cumulative ranking from 10 separate rankings October 2009 by Marco Geraci http://rpc264.cs.man.ac.uk/VIA/index.php/VIA25

  6. M Degli Esposti, presented at the second conference of Via-academy, 18 September 2010

  7. From ‘top Italian scientists’: ranking of Italian institutes La Stampa 30/8/10 400 TIS La Repubblica 5/1/2011 1230 TIS M Degli Esposti, December 31, 2010 – retrieved from: http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx

  8. The media resonance of ‘top Italian scientists’

  9. Italian biochemists and their h-index Note Valori di h-index sono stati calcolati cosi’: X, con Google Scholar add-standard, con iniziale, ristretto alle aree ‘chemistry’, ‘biology’ and ‘medicine’ Y, con ISI Web of Science, In genere ‘all years’ nei settori di bio-medicina (non clinici) e chimica (e.g. medicinal chemistry) mean for full professors = 19 Mauro Degli Esposti La Voce.it, 14-9-10 http://www.topitalianscientists.org/downloads/Manualetto_del_Ricercatore_Provetto_ver1.2.pdf

  10. Home - in Italy (50) Abroad (50) >70 >70 Italian scientists: home and abroad TIS Reports 001 http://tisreports.com/fullreport/1-Top_100_-_home___abroad.aspx

  11. Top Italian Scientists – towards the saturation of names? 83% of the top100 work in Bio-medicine! M Degli Esposti, December 31, 2010 – retrieved from: http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx

  12. 200 Top Italian scientists abroad (total = 363) 150 100 number 50 0 UK USA Spain France Austria Vatican Sweden Belgium Canada Germany Swizerland Other nations M Degli Esposti (2010), retrievedJanuary 3, 2011 from: http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx

  13. Nel 2006 Italia = 93000 ricercatori Fonte : OCSE

  14. 500 450 top scienziati per discipline 400 350 300 numero di scienziati 250 200 150 100 50 0 fisica chimica biologia medicina economia astrofisica informatica technology matematica farmacologia Areas of relative excellence for Italy M Degli Esposti, December 31, 2010 – retrieved from: http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx

  15. TIS Fonte : ISTAT

  16. Ranking di universita’ statali e non – problems L’università italiana ha finito anche la frutta di Andrea Aparo IL Fatto Quotidiano, 12 Novembre 2010 Ricercatori italiani uguale fannulloni? di Pietro Greco Scienza in Rete (http://www.scienzainrete.it) 04/05/2009

  17. Marco Geraci e Mauro Degli Esposti, unpublished results, October 2010 Ranking for Italian universities – various and different

  18. In depth analysis of all ranking – recent advances GRAND VIA - a new meta-ranking combining 5 international and 4 national ones From co-variance patterns – principal componentsanalysis, PCA (Pearson, 1901) Marco Geraci e Mauro Degli Esposti, November 2010, in preparation

  19. Vision Ranking of Italian universities: 2 principal components Principal component analysis of different rankings of Italian universities Grand VIA Top Italian Scientists National performance International impact and visibility Marco Geraci e Mauro Degli Esposti, November 2010 in preparation

  20. Vision Ranking from ‘top Italian scientists’ – better than others! Computed over 4 different international rankings Marco Geraci e Mauro Degli Esposti, November 2010

  21. Evaluation landscape of state Italian universities Marco Geraci e Mauro Degli Esposti, submitted

  22. Landscape of state Italian universities - all Ferrara Marco Geraci e Mauro Degli Esposti 2010, submitted…

  23. Ferrara university – ranking among top institutes in Italy:before15-12 and after systematic analysis 31-12 Total TIS = 15 + 11 new TIS = 26

  24. and after systematic analysis Ranking = 10 + 12 new TIS = 27 M Degli Esposti, January 5, 2011 - retrieved from: http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx Ferrara university – ranking among state universities:before Ranking = 15 Total TIS = 15 M Degli Esposti, December 15, 2010 – retrieved from: http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx

  25. NOT ISI WoS! M Degli Esposti, September, 2010 – retrieved from: Manueletto del Ricercatore Provetto Different Area – different impact(publications and citations) • Normalization? By which criteria? • Which database to use for reference?

  26. Scientometrics 73 (2007) The normalization factor follows a ‘stretched exponential’; it becomes bigger by increasing the number of published articles For scientists with less than 50 published articles, the field-specific differences are small

  27. h-index - discrepancies between different databases Note Valori di h-index sono stati calcolati cosi’: X, con Google Scholar add-standard, con iniziale, ristretto all’area ‘egineering and mathematics’ Y, con ISI Web of Science, all years, ristretto ai settori matematici Matematici con nomi Compositi (e.g. De Giorgi) sono esclusi - ISI non li trova! NOTA: usando Google Scholar si normalizza gia’ le citazioni dei matematici rispetto a quelle che si computano con ISI WoS. Questo risulta in una normalizzazione automatica di un fattore = 1.67, Simile al fattore di nromalizzatione proposto da Iglesias (2007) ca. = 1.83 Mauro Degli Esposti – Via-academy.org, 6-9-2010

  28. 25 ISI 20 Google Scholar 15 Average h-index Add-on PorP 10 ISI WoS 5 0 database Discrepancies between different databases for h-index: the problem with chemistry All chemists of Ferrara university Correlation factor = 1.38 Mauro Degli Esposti, 21-12-2010

  29. average h-index Ferrara departments 40 y = 0.9086x 35 2 R = 0.9325 30 25 Add-On (full name) 20 15 10 5 0 0 5 10 15 20 25 30 35 40 PorP (initial) Average h-index Total citations Number of TIS Mauro Degli Esposti, 5-1-2011

  30. Fonte : National Science Board

  31. Correlation, R2 = 0.72 6 4 Publications per year x 10-3 2 Relative disease burden (Log %) Biomedicine: area variation of publication rate Neoplasms %articles that are patent-associated[

  32. Ordine alfabetico 5-1-2011

  33. Fonte : OCSE

  34. MERIT -evaluation Let’s the discussion begin!

  35. Fondazione CRUI luglio 2010 CAF = Common Assessment Framework H-index, pagina 65 (su 85)

  36. Evaluate university institutions and individuals:The VIA-Ferrara model A 3-step new evaluation model TIME FRAME: end of 2010, beginning of 2011 Mauro Degli Esposti, 6 December 2010

  37. Evaluate university institutions and individuals:The Manchester model Total weight Publications = 40% of RQI Mauro Degli Esposti, 6 December 2010

  38. Evaluate university institutions and individuals:The Manchester model Research Quality Indicator = RQI PROBLEMS: 1. Evaluation of Publication quality by Internal Committee – prone to politics! 2. No consideration of international standing and individual’s scientific impact, e.g. h-index Before RAE 2009: Category - weight Esteems 5 Publications 75 Environment* 25 *including research income Mauro Degli Esposti, 6 December 2010

  39. Evaluate university institutions and individuals:The VIA-Ferrara model A 3-step new evaluation model • 2. Ferrara • All scientists to provide by e-mail: • a list with their top 4 papers in the last 2 years; • the number of Ph.D. and Master students supervised; • the total grant income obtained from external sources. • Via-academy • Quantitative evaluation of the scientific impact of all Ferrara scientists based on the systematic analysis of h-index. • Area-specific analysis of h-index using average values of top Italian scientists as reference TIME FRAME: end of 2010 3. Combine the analysis 1 with data in 2 to define the basic format (like in Manchester) and quantify individual profiles - with 1 at least counting for 50% Mauro Degli Esposti, 6 December 2010

  40. Simple model for auto-evaluation in specific disciplines within a university Set a threshold of intrinsic (local) excellence by applying this formula: Threshold = average + SD + 1 (in absolute value of h-index) For example, Ferrara chemists: Threshold = 15.6 + 7.5 + 1 = 24 Questions?

  41. Italian scientists: home and away Media dei valori di h-index per gruppi di scienziati ed accademici italiani che lavorano in Italia. I valori sono simili a quelli degli Italiani che lavorano all’estero. Nota: la barra corrisponde alla deviazione standard. Valore soglia per entrare in lista Mauro Degli Esposti – La Voce.it, 14-9-2010

  42. Su un totale di 1188 scienziati nella TIS al 15-12-2010, Di cui 834 in Italia – escluso CNR ed INAF M Degli Esposti, December 15, 2010 – retrieved from: http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx

More Related