Benchmarking Universities Worldwide: 
This presentation is the property of its rightful owner.
Sponsored Links
1 / 44

Benchmarking Universities Worldwide:  Existing Results and Future Efforts of  Academic Ranking of World Universities PowerPoint PPT Presentation


  • 126 Views
  • Uploaded on
  • Presentation posted in: General

Benchmarking Universities Worldwide:  Existing Results and Future Efforts of  Academic Ranking of World Universities. Presented By Dr. Ying CHENG Institute of Higher Education, Shanghai Jiao Tong University, China Observatoire des Sciences et des Techniques (OST), France

Download Presentation

Benchmarking Universities Worldwide:  Existing Results and Future Efforts of  Academic Ranking of World Universities

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Benchmarking Universities Worldwide: Existing Results and Future Efforts of  Academic Ranking of World Universities

Presented By Dr. Ying CHENG

Institute of Higher Education, Shanghai Jiao Tong University, China

Observatoire des Sciences et des Techniques (OST), France

December 12, 2007Université Libre de Bruxelles, Belguim


Outline

Outline

  • Purpose

  • Methodologies & Results

  • Problems and Discussion

  • Ranking by Broad Subject Fields

  • Future Efforts


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Purpose of ARWU


Dream of chinese for wcu

Dream of Chinese for WCU

  • World-class university (WCU) is a dream for generations of Chinese. It’s not only for pride, but also for the future of China.

  • Recently, Chinese government has launched several initiatives for research universities. The best-known one is specially designed to build WCU (985 Project).


Goals of top chinese universities

Goals of Top Chinese Universities

  • Many top Chinese universities have setup their strategic goals as WCU.

  • Most of them have also set time tables for reaching the goal of WCU. For example:

    2016 for Peking University

    2020 for Tsinghua University


Questions about wcu

Questions About WCU

  • Is there a clear definition for WCU?

  • How many WCU should there be in the world?

  • What are the positions of top Chinese universities in the world?

  • How can Chinese universities improve themselves to reach the goal of WCU?


Academic ranking of world universities

Academic Ranking of World Universities

  • Our original purpose of doing the Academic Ranking of World Universities (ARWU) was to find out the position of Chinese universities in the world and the gap between them and WCU.

  • ARWU was put on the internet upon the encouragement of colleagues from all over the world. There have been an average of 2000 visitors every day since 2003.


Features of arwu

Features of ARWU

  • ARWU uses a few carefully selected, objective criteria and internationally comparable data that everyone could verify in some way.

  • It has been carried out by a ranking team (chaired by Professor Nian Cai LIU) in the Institute of Higher Education of Shanghai Jiao Tong University for their academic interests.

  • It has been done independently without any financial support from any sources outside the Institute of Higher Education.


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Methodologies & Results of ARWU


Selection of universities

Selection of Universities

  • Any university that has any Nobel Laureates, Fields Medals, Highly Cited Researchers, or papers published in Nature or Science.

  • Major universities of every country with significant amount of papers indexed by Thomson.

    • Number of universities scanned: >2000

    • Number of universities actually ranked: >1000

    • Number of ranked universities on our web: 500


Ranking criteria and weights

Criteria

Indicator

Code

Weight

Quality of

Alumni of an institution winning Nobel

Alumni

10%

Education

Prizes and Fields Medals

Staff of an institution winning Nobel

Award

20%

Prizes and Fields Medals

Quality of Faculty

Highly cited researchers in 21 broad

HiCi

20%

subject categories

Articles published in Nature and Science*

N&S

20%

Research Output

Articles in S

CIE

SSCI

SCI

20%

Academic performance

with respect to

Size of Institution

Size

1

0%

the size of an institution

Total

100%

For institutions specialized in humanities and social sciences such as London

School of Economics

,

s.

N&S is not consider

ed, and the weight of N&S is

relocated to other indicator

Ranking Criteria and Weights


Definition of indicator alumni

Definition of Indicator: Alumni

  • The total number of the alumni of an institution winning Nobel Prizes and Fields Medals.

  • Alumni are defined as those who obtain bachelor, Master’s or doctoral degrees from the institution.

  • Different weights are set according to the periods of obtaining degrees. The weight is 100% for alumni of 1991-2000, 90% for alumni of 1981-1990, 80% for alumni of 1971-1980, and so on.

  • If a person obtains more than one degrees from an institution, the institution is considered once only.


Definition of indicator award

Definition of Indicator: Award

  • The total number of the staff of an institution winning Nobel prizes in physics, chemistry, medicine and economics and Fields Medal in Mathematics.

  • Staff is defined as those who work at an institution at the time of winning the prize.

  • Different weights are set according to the periods of winning the prizes. The weight is 100% for winners since 2001, 90% for winners in 1991-2000, 80% and so on.

  • If a winner is affiliated with more than one institution, each institution is assigned the reciprocal of the number of institutions.

  • For Nobel prizes, if a prize is shared by more than one person, weights are set for winners according to their proportion of prize.


Definition of indicator hici

Definition of Indicator: HiCi

  • The number of highly cited researchers in 21 broad subject categories in life sciences, medicine, physical sciences, engineering and social sciences.

  • The definition of categories and detailed procedures can be found at the website of Institute of Scientific Information.

  • The total number of HiCi is about 6000, about 4000 of which is university staff.


Definition of indicator n s

Definition of Indicator: N&S

  • The annual average number of articles published in Nature and Science in the past five years.

  • To distinguish the order of author affiliation, a weight of 100% is assigned for corresponding author, 50% for first author (second author if the first author is the same as corresponding author), 25% for the next author, and 10% for other authors.

  • Only publications of article type are considered.


Definition of indicator sci

Definition of Indicator: SCI

  • Total number of articles indexed in Science Citation Index-expanded (SCIE) and Social Science Citation Index (SSCI) in the past year.

  • A weight of 2 is assigned to articles indexed in SSCI to compensate the bias against humanities and social sciences.

  • Only publications of article type are considered.


Definition of indicator size

Definition of Indicator: Size

  • The sub-total scores of the above five indicators divided by the number of full-time equivalent academic staff.

  • If the number of academic staff for institutions of a country cannot be obtained, the total scores of the above five indicators is used.

  • For ranking 2007, the number of full-time equivalent academic staff is obtained for institutions in USA, China, Australia, Italy, Netherlands, Sweden, and Belgium etc.


Main sources of data

Main Sources of Data

  • Nobel laureates:

    http://www.nobelprize.org

  • Fields Medals:

    http://www.mathunion.org/medals/

  • Highly-cited researchers:

    http://www.isihighlycited.com

  • Articles published in Nature and Science:

    http://www.isiknowledge.com

  • Articles indexed in SCIE and SSCI:

    http://www.isiknowledge.com


Results of arwu

Results of ARWU

  • http://ed.sjtu.edu.cn/ranking.htm

    • Top 500 universities in the world

    • Top 100 universities in North and Latin America

    • Top 100 universities in Asia/Oceania

    • Top 100 universities in Europe

    • Statistics of top universities by region and country

    • Percentage distribution of top universities by country as compared with the share of global population and GDP


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Problems and Discussion of ARWU


Methodological education and service

Methodological: Education and Service

  • Educationis the basic function of any university, however, it would be impossible to rank the quality of education due to the huge differences among the national systems.

  • Contribution to the national economic developmentis becoming increasingly important for universities, however, it is almost impossible to obtain internationally comparable indicators and data.

  • The academic orresearch performanceof universities, a good indication of their reputation, can be ranked internationally.


Methodological humanities social sciences

Methodological: Humanities & Social Sciences

  • Many well-known institutions specialized in humanities and social sciences are ranked relatively low.

  • Since 2004, the indicator of N&S is not considered for institutions specialized in humanities and social sciences, its weight is relocated to other indicators.

  • Since 2005, a weight of 2 for articles indexed by SSCI is considered.

  • Nevertheless, if a university specialized in social sciences and humanities had Nobel Laureates in economics and Highly Cited Researchers in social sciences, it should have good standing.


Methodological language bias

Methodological: Language Bias

  • English is the language of international academic community.

  • Almost any ranking based on academic performance will be biased towards institutions in English-speaking countries.

  • One possible solution: papers published in non-native languages are offered a special weight.

  • Another possible solution: normalization of total articles by the proportion of journal editors of each country.


Methodological award and alumni

Methodological: Award and Alumni

  • Universities which started after 1911 do not have a fair chance.

  • Disciplines not related to the awarding fields do not have a fair chance. Other important awards include Abel, Pulitzer, Turing, Tyler, Pritzker, etc.

  • Institutions for winning awards and those for doing the researches may not be the same.

  • Institutions for obtaining degrees and those for pursuing the studies may not be the same.


Methodological per capita performance

Methodological: Per Capita Performance

  • The weight of the Size indicator for per capita performance is rather low. Large institutions have relatively high positions in the ranking.

  • However, it’s very difficult to obtain internationally comparable data on the number of academic staff.

  • The types of academic staff: such as purely teaching staff, teaching and research staff, purely research staff.

  • The ranks of academic staff: such as professor, associate professor, reader, lecturer, research scientist etc.


Technical attributions

Technical: Attributions

  • Many universities havemore than one commonly used names: such as Virginia Tech and Virginia Polytechnic and State University.

  • Variations due to translation: such as Univ Koln and Univ Cologne, Univ Vienna and Univ Wien.

  • Abbreviated names: such as ETH Zurich for Swiss Federal Institute of Technology Zurich.


Technical definition of institution

Technical: Definition of Institution

  • University systems: such as Univ California system, Univ London system.

  • Affiliated institutions and research organizations: such as Ecole Polytechnique Montreal (affiliated to University of Montreal), CNRS Labs (affiliated to French universities).

  • Teaching and affiliated Hospitals: complex!

  • Our answer: according to author’s expression.


Other technical problems

Other Technical Problems

Merging, splitting, inheriting, discontinuing, name-changing of institutions such as:

  • Univ Kwazulu-Natal in South Africa merged from Univ Natal and Univ Durban-Westville.

  • University of Innsbruck in Austria splitted into Univ Innsbruck and Innsbruck Medical Univ.

  • Humboldt Univ Berlin and Free Univ Berlin inheriting the Nobel Prizes of the Berlin University before world war II.


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Ranking by Broad Subject Fields

(ARWU-FIELD)


Purpose of arwu field

Purpose of ARWU-FIELD

  • Requests for Ranking of World Universities by

    • Broad subject fields or schools, colleges and

    • Subject fields or programs, departments

  • Many top Chinese universities want to learn their positions in the world by broad subject fields or disciplines.


Definition of broad subject fields

Definition of Broad Subject Fields

  • Natural Sciences and Mathematics (SCI)

  • Engineering/Technology and Computer Sciences (ENG)

  • Life and Agriculture Sciences (LIFE)

  • Clinical Medicine and Pharmacy (MED)

  • Social Sciences (SOC)

  • Arts and humanities are not ranked


Arwu field indicators and weights

ARWU-FIELD Indicators and Weights


Changes in indicators and definition

Changes in Indicators and Definition

  • N&S in ARWU is not used in ARWU-FIELD.

  • TOP is the percentage of articles published in thetop 20%journals of each broad subject field.

  • Fund is the total engineering-related research expenditures. It’s used only forENGranking.

  • Alumni and Award since 1951 are used for all rankings fields except ENG.

  • PUB is the total number of articles indexed by Thomson in the past year.


Results of arwu field

Results of ARWU-FIELD

  • http://ed.sjtu.edu.cn/ranking.htm

    • Top 100 universities in SCI

    • Top 100 universities in ENG

    • Top 100 universities in LIFE

    • Top 100 universities in MED

    • Top 100 universities in SOC

    • Statistics of top universities by region & country

    • List of top universities by number of top fields


Special problems in arwu field

Special Problems in ARWU-FIELD

  • It’s difficult to obtain data on engineering-related research expenditures and make them comparable. For 2007, Fund was obtained only for US and Canadian universities.

  • It’s difficult to separate the Nobel Laureates in Physiology or Medicine. They are used in bothLIFE and MED ranking.


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Future Efforts of ARWU


Study the methodological and technical problems

Study the Methodological and Technical Problems

Welcome idea and suggestions!


Update arwu annually

Update ARWU Annually

  • In order to observe changes of Universities

  • In order to monitor countries’ performance

  • Serve as the basis of related academic studies and policy analysis


Provide different rankings

Provide Different Rankings

  • Ranking by per capita performance

    • Based on reliable and international comparable data of academic staff

  • Ranking of universities specialized or strong in engineering, medicine, etc.

    • Based on the classification of world universities


Establish database of wcu

Establish Database of WCU

  • Basic Characteristics

    • Students, Staff, Program, Budget, …

  • Other Indicators for International Comparison

    • R&D Expenditure, Bibliometric Indicators, International Academic Awards, …

  • Data at College/Department Level


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Final Remarks


Controversy of ranking

Controversy of Ranking

  • Any ranking is controversial and no ranking is absolutely objective.

  • University rankings become popular in many countries. Whether we agree or not, ranking systems clearly are here to stay.

  • The key issue then becomes how to improve ranking systems for the benefits of higher education.


Use of ranking

Use of Ranking

  • Rankings are tools for different purpose.

  • Rankings should be used with cautions. Their methodologies must be read carefully before reporting or using their results.

  • Rankings should be used in combination with other types of evaluation whenever possible.


Benchmarking universities worldwide existing results and future efforts of academic ranking of world universities

Thanks for Your Attention!

http://ed.sjtu.edu.cn/ranking.htm

E-mail: [email protected]


  • Login