Towards a new model for evaluating the merit of Italian
This presentation is the property of its rightful owner.
Sponsored Links
1 / 42

Mauro Degli Esposti PowerPoint PPT Presentation


  • 59 Views
  • Uploaded on
  • Presentation posted in: General

Towards a new model for evaluating the merit of Italian academic scientists and their institutions. Mauro Degli Esposti. Ferrara, Jan 2011. A new analysis of Italian universities – the start. From The Times October 9, 2008 Whatever happened to the University of Bologna?

Download Presentation

Mauro Degli Esposti

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Mauro degli esposti

Towards a new model for evaluating the merit of Italian

academic scientists and their institutions

Mauro Degli Esposti

Ferrara, Jan 2011


Mauro degli esposti

A new analysis of Italian universities – the start

From The Times October 9, 2008

  • Whatever happened to the University of Bologna?

  • The University of Bologna, which received its charter from Frederick I Barbarossa in 1158, was probably the first university founded in the Western world. It became a renowned home to scholars of great distinction in both the sciences and the humanities. The Times Higher Education-QS World University Rankings survey was published this week. It rates the performance of universities according to the quality of their research, the employment rates of graduates and their international profile.

  • The University of Bologna does not feature in the top 100!

Harvard = 1, Cambdrige = 3, Oxford = 4, UCL = 7...

Manchester = 29 – now 26

M Degli Esposti, presented at the first conference of Via-academy, 21 September 2009

http://via-academy.org/Relazione_Prima_Conferenza_della_VIA_a_Manchester.aspx


Mauro degli esposti

Diritto di cronaca

12/10/2009

Un ateneo per scappare

di FLAVIA AMABILE

MERIT – a new approach to evaluation

The members of the Via-academy network who work in UK have been selected

on the basis of merit, but educated in Italy.

The number of academic links of Via-members with Italian institution will be

proportional to the relative teaching capacity of these institutions – i.e. they

will be an output measure of their quality and relative standing.

M Degli Esposti, presented at the first conference of Via-academy, 21 September 2009

http://via-academy.org/Relazione_Prima_Conferenza_della_VIA_a_Manchester.aspx


Mauro degli esposti

by Marco Geraci

M Degli Esposti, presented at the first conference of Via-academy, 21 September 2009


Mauro degli esposti

VIA25 - cumulative ranking from 10 separate rankings

October 2009

by Marco Geraci

http://rpc264.cs.man.ac.uk/VIA/index.php/VIA25


Mauro degli esposti

M Degli Esposti, presented at the second conference of Via-academy, 18 September 2010


Mauro degli esposti

From ‘top Italian scientists’: ranking of Italian institutes

La Stampa 30/8/10

400 TIS

La Repubblica 5/1/2011

1230 TIS

M Degli Esposti, December 31, 2010 – retrieved from:

http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx


Mauro degli esposti

The media resonance of ‘top Italian scientists’


Italian biochemists and their h index

Italian biochemists and their h-index

Note

Valori di h-index sono

stati calcolati cosi’:

X, con Google Scholar

add-standard, con iniziale,

ristretto alle aree ‘chemistry’,

‘biology’ and ‘medicine’

Y, con ISI Web of Science,

In genere ‘all years’

nei settori di bio-medicina

(non clinici) e chimica

(e.g. medicinal chemistry)

mean for full professors = 19

Mauro Degli Esposti La Voce.it, 14-9-10

http://www.topitalianscientists.org/downloads/Manualetto_del_Ricercatore_Provetto_ver1.2.pdf


Italian scientists home and abroad

Home - in Italy (50)

Abroad (50)

>70

>70

Italian scientists: home and abroad

TIS Reports 001

http://tisreports.com/fullreport/1-Top_100_-_home___abroad.aspx


Mauro degli esposti

Top Italian Scientists – towards the saturation of names?

83% of the top100 work in

Bio-medicine!

M Degli Esposti, December 31, 2010 – retrieved from:

http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx


Mauro degli esposti

200

Top Italian scientists abroad

(total = 363)

150

100

number

50

0

UK

USA

Spain

France

Austria

Vatican

Sweden

Belgium

Canada

Germany

Swizerland

Other nations

M Degli Esposti (2010), retrievedJanuary 3, 2011 from:

http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx


Mauro degli esposti

Nel 2006 Italia = 93000 ricercatori

Fonte : OCSE


Mauro degli esposti

500

450

top scienziati per discipline

400

350

300

numero di scienziati

250

200

150

100

50

0

fisica

chimica

biologia

medicina

economia

astrofisica

informatica

technology

matematica

farmacologia

Areas of relative excellence for Italy

M Degli Esposti, December 31, 2010 – retrieved from:

http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx


Mauro degli esposti

TIS

Fonte : ISTAT


Mauro degli esposti

Ranking di universita’ statali e non – problems

L’università italiana ha finito anche la frutta

di Andrea Aparo

IL Fatto Quotidiano, 12 Novembre 2010

Ricercatori italiani uguale fannulloni?

di Pietro Greco

Scienza in Rete (http://www.scienzainrete.it)

04/05/2009


Mauro degli esposti

Marco Geraci e Mauro Degli Esposti, unpublished results, October 2010

Ranking for Italian universities – various and different


Mauro degli esposti

In depth analysis of all ranking – recent advances

GRAND VIA - a new meta-ranking combining 5 international and 4 national ones

From co-variance patterns – principal componentsanalysis, PCA (Pearson, 1901)

Marco Geraci e Mauro Degli Esposti, November 2010, in preparation


Mauro degli esposti

Vision

Ranking of Italian universities: 2 principal components

Principal component analysis of different rankings of Italian universities

Grand VIA

Top Italian Scientists

National performance

International impact and visibility

Marco Geraci e Mauro Degli Esposti, November 2010 in preparation


Mauro degli esposti

Vision

Ranking from ‘top Italian scientists’ – better than others!

Computed over 4 different international rankings

Marco Geraci e Mauro Degli Esposti, November 2010


Mauro degli esposti

Evaluation landscape of state Italian universities

Marco Geraci e Mauro Degli Esposti, submitted


Mauro degli esposti

Landscape of state Italian universities - all

Ferrara

Marco Geraci e Mauro Degli Esposti 2010, submitted…


Ferrara university ranking among top institutes in italy before 15 12

Ferrara university – ranking among top institutes in Italy:before15-12

and after systematic analysis 31-12

Total TIS = 15

+ 11 new TIS = 26


Ferrara university ranking among state universities before

and after systematic analysis

Ranking = 10

+ 12 new TIS = 27

M Degli Esposti, January 5, 2011 - retrieved from:

http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx

Ferrara university – ranking among state universities:before

Ranking = 15

Total TIS = 15

M Degli Esposti, December 15, 2010 – retrieved from:

http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx


Different area different impact publications and citations

NOT ISI WoS!

M Degli Esposti, September, 2010 – retrieved from: Manueletto del Ricercatore Provetto

Different Area – different impact(publications and citations)

  • Normalization? By which criteria?

  • Which database to use for reference?


Mauro degli esposti

Scientometrics 73 (2007)

The normalization factor

follows a ‘stretched exponential’;

it becomes bigger by increasing the number of published articles

For scientists with less than

50 published articles, the field-specific differences are small


H index discrepancies between different databases

h-index - discrepancies between different databases

Note

Valori di h-index sono

stati calcolati cosi’:

X, con Google Scholar

add-standard, con iniziale,

ristretto all’area

‘egineering and mathematics’

Y, con ISI Web of Science,

all years, ristretto

ai settori matematici

Matematici con nomi

Compositi (e.g. De Giorgi)

sono esclusi - ISI non li trova!

NOTA: usando Google Scholar si normalizza gia’ le citazioni dei matematici

rispetto a quelle che si computano con ISI WoS.

Questo risulta in una normalizzazione automatica di un fattore = 1.67,

Simile al fattore di nromalizzatione proposto da Iglesias (2007) ca. = 1.83

Mauro Degli Esposti – Via-academy.org, 6-9-2010


Mauro degli esposti

25

ISI

20

Google Scholar

15

Average h-index

Add-on

PorP

10

ISI WoS

5

0

database

Discrepancies between different databases for h-index:

the problem with chemistry

All chemists of Ferrara university

Correlation factor = 1.38

Mauro Degli Esposti, 21-12-2010


Mauro degli esposti

average h-index Ferrara departments

40

y = 0.9086x

35

2

R

= 0.9325

30

25

Add-On (full name)

20

15

10

5

0

0

5

10

15

20

25

30

35

40

PorP (initial)

Average h-index

Total citations

Number of TIS

Mauro Degli Esposti, 5-1-2011


Mauro degli esposti

Fonte : National Science Board


Biomedicine area variation of publication rate

Correlation, R2 = 0.72

6

4

Publications per year x 10-3

2

Relative disease burden (Log %)

Biomedicine: area variation of publication rate

Neoplasms

%articles that are

patent-associated[


Mauro degli esposti

Ordine alfabetico

5-1-2011


Mauro degli esposti

Fonte : OCSE


Mauro degli esposti

MERIT -evaluation

Let’s the discussion begin!


Fondazione crui luglio 2010

Fondazione CRUI luglio 2010

CAF = Common

Assessment

Framework

H-index, pagina 65 (su 85)


Evaluate university institutions and individuals the via ferrara model

Evaluate university institutions and individuals:The VIA-Ferrara model

A 3-step new evaluation model

TIME FRAME: end of 2010, beginning of 2011

Mauro Degli Esposti, 6 December 2010


Evaluate university institutions and individuals the manchester model

Evaluate university institutions and individuals:The Manchester model

Total weight

Publications =

40% of RQI

Mauro Degli Esposti, 6 December 2010


Evaluate university institutions and individuals the manchester model1

Evaluate university institutions and individuals:The Manchester model

Research Quality Indicator = RQI

PROBLEMS:

1. Evaluation of Publication quality by

Internal Committee – prone to politics!

2. No consideration of international standing

and individual’s scientific impact, e.g. h-index

Before RAE 2009:

Category - weight

Esteems5

Publications75

Environment*25

*including research income

Mauro Degli Esposti, 6 December 2010


Evaluate university institutions and individuals the via ferrara model1

Evaluate university institutions and individuals:The VIA-Ferrara model

A 3-step new evaluation model

  • 2. Ferrara

  • All scientists to provide by e-mail:

  • a list with their top 4 papers in the last 2 years;

  • the number of Ph.D. and Master students supervised;

  • the total grant income obtained from external sources.

  • Via-academy

  • Quantitative evaluation of the scientific impact of all Ferrara scientists based on the systematic analysis of h-index.

  • Area-specific analysis of h-index using average values of top Italian scientists as reference

TIME FRAME: end of 2010

3. Combine the analysis 1 with data in 2 to define the basic format (like in

Manchester) and quantify individual profiles - with 1 at least counting for 50%

Mauro Degli Esposti, 6 December 2010


Questions

Simple model for auto-evaluation in specific disciplines within a university

Set a threshold of intrinsic (local) excellence by applying this formula:

Threshold = average + SD + 1 (in absolute value of h-index)

For example, Ferrara chemists:

Threshold = 15.6 + 7.5 + 1 = 24

Questions?


Italian scientists home and away

Italian scientists: home and away

Media dei valori di h-index per gruppi di scienziati ed accademici italiani

che lavorano in Italia. I valori sono simili a quelli degli Italiani che lavorano all’estero.

Nota: la barra corrisponde alla deviazione standard.

Valore soglia

per entrare in lista

Mauro Degli Esposti – La Voce.it, 14-9-2010


Mauro degli esposti

Su un totale di 1188 scienziati nella TIS al 15-12-2010,

Di cui 834 in Italia – escluso CNR ed INAF

M Degli Esposti, December 15, 2010 – retrieved from:

http://www.topitalianscientists.org/Top_italian_scientists_VIA-Academy.aspx


  • Login