Isi research services group
This presentation is the property of its rightful owner.
Sponsored Links
1 / 40

ISI Research Services Group: PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on
  • Presentation posted in: General

ISI Research Services Group:. Analytical tools for Research Evaluation. Outline. Research evaluation: Unique features of Web of Knowledge Journal Selection Quantitative tools Cui bono: What sectors use these analyses Citation Analysis – Issues and Caveats Free Stuff!!.

Download Presentation

ISI Research Services Group:

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Isi research services group

ISI Research Services Group:

Analytical tools for Research Evaluation


Outline

Outline

  • Research evaluation:

    • Unique features of Web of Knowledge

    • Journal Selection

    • Quantitative tools

  • Cui bono: What sectors use these analyses

  • Citation Analysis – Issues and Caveats

  • Free Stuff!!

ISI Research Services Group Nancy K. Bayers


The web of knowledge is

The Web of Knowledge is…

  • An integrated, web-based environment providing high quality content and the tools to access, analyze, and manage scientific and scholarly information

  • Incorporating new technologies (linking, searching, and personalization) to integrate, extend, and enhance the researchenvironment

ISI Research Services Group Nancy K. Bayers


Isi research services group

Web of Knowledgesm

E-Journal Full-Text

A Free Resource!!

Evaluated, Selected Web Content

E-Patent Full-Text

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • A.Journal Selection Process

  • “A cost effective index index must restrict its

  • coverage, as nearly as possible to only those

  • items that researchers are likely to find useful.”

  • -Eugene Garfield, Ph. D.

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • … Journal Selection

    • 1.Basic Journal Publishing Standards

    • 2.Editorial Content

    • 3.International Diversity

    • 4.Citation Analysis

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

… Journal Selection

1.)Publishing Standards

a.) Timeliness

b.) Editorial Conventions

c.) English Language Bibliographic Information

d.) Peer Review

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a.) Timeliness of Publication

  • A journal must be published according to its stated publication schedule to be considered for coverage in the ISI database.

  • ISI must receive three on-time issues in sequence.

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ... Journal Selection

  • 1. Basic Journal Publishing Standards

    • a.) Timeliness of Publication

    • b.) International Editorial Conventions

    • To optimize retrievability:

    • - Informative Journal Titles

      • - Descriptive Article Titles

      • - Full Address Information for Every Author

      • - Complete Bibliographic Information for

      • All Cited References

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a.) Timeliness of Publication

b.) International Editorial Conventions

c.) English Language Bibliographic Information

- Article Titles

- Author Names and Addresses

- Cited References

- Abstracts and Keywords

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a. Timeliness of Publication

b. International Editorial Conventions

c. English Language Bibliographic Information

d. Peer Review

Application of the peer review process is another indication of the journal’s standards, the overallquality of research presented, and the completeness of cited references.

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ... Journal Selection

  • 2.) Editorial Content

    • Do we need this new journal in the database, or is the category already well covered?

    • Does this journal offer something new?

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ... Journal Selection

  • 3.) International Diversity

    • Are the authors, editors, and editorial advisory

      • board members from around the world?

    • Does this journal reflect the global context

      • in which scientific research takes place?

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ...3. International Diversity

  • Regional Journals:

  • To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI also seeks to cover the best regional journals as well.

  • ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    • ... Journal Selection

    • 4. Citation Analysis

      • Expert use of citation data help ISI identify

      • important, influential, and useful publications.

      • ISI measuresuse by analyzing citation data.

    ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    • ... Journal Selection

    • ... 4. Citation Analysis

      • New Journals

        • Has the past work of authors and editorial

        • advisory board members received citations?

    • Established Journals

    • Impact Factor

    ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    ... Journal Selection

    ...4. Citation Analysis

    Impact Factor:

    The average number of times recent articles

    in a journal were cited in a particular year.

    Nature Genetics:

    2000 Impact factor: 30.910

    ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    Journal Citation Reports (JCR) -- 2001 Science Edition

    Impact Factor Calculation

    Journal:Nature Genetics

    Impact Factor:29.600

    Cites in 2001 to articles published in :2000 = 6,167

    1999 = 7,538

    00+99 = 13,705

    Number of articles published in:2000 = 253

    1999 = 210 00+99 = 463

    Calculation:

    Cites to recent articles 13,705 = 29.600

    Number of recent articles 463

    ISI Research Services Group Nancy K. Bayers


    Research evaluation

    Research Evaluation

    • Web of Knowledge: unique features

      • cited references -- footnotes

      • all author names

      • all author addresses

    • Count

      • papers -- output

      • citations -- influence or impact

    • Baselines

    ISI Research Services Group Nancy K. Bayers


    Cui bono

    Cui bono?

    • Policymakers

    • Research directors & administrators

    • Researchers

    • Publishers

    • Information specialists

    ISI Research Services Group Nancy K. Bayers


    Words to the wise

    Words to the wise...

    • Citation analysis is a fair evaluative tool in those fields where publication in the journal literature is the main vehicle of communication.

    • Need multiple indicators. Citation analysis is only a quantitative

      supplement to other measures of research evaluation,

      including peer review

    • There are always exceptions to everything.

    ISI Research Services Group Nancy K. Bayers


    Citation analysis issues and caveats

    Citation analysis: issues and caveats

    • Field publication behavior differences

    • How to handle...

      • Self cites and negative cites

      • Reviews and methods papers

      • English language bias

    • Skewed distributions

    • Relative measures/baselines

    • Confidence levels

    • What citations do not reflect

    ISI Research Services Group Nancy K. Bayers


    Field citation rates average cites 1997 2001

    Field citation rates: average cites 1997-2001

    • Cell & Developmental Biology 15.9

    • Molecular Biology 10.6

    • Biochemistry & Biophysics9.1

    • Neuroscience7.3

    • Chemistry & Analysis5.9

    • Physics4.4

    • Earth Science3.5

    • Materials Science2.1

    • Mathematics1.3

    ISI Research Services Group Nancy K. Bayers


    Field publication rates size

    Field publication rates/size

    Field% papers in world

    • Clinical Medicine24%

    • Chemistry14%

    • Physics12%

    • Biology & Biochemistry 8%

    • Plant & Animal Science 6%

    • Materials Science 4%

    • Neuroscience 4%

    • Geosciences 3%

    • Mathematics 2%

    • Computer Science 1%

    ISI Research Services Group Nancy K. Bayers


    Self cites and negative cites

    Self-cites and negative cites

    SELF CITES

    • Cumulative nature of individual research

    • a natural and acceptable procedure if not excessive

    • Approximately 25% in biomedicine

    • 16% in plant physiology

    • 17.5% in neurobiology

      NEGATIVE CITES

    • Only 7%

    ISI Research Services Group Nancy K. Bayers


    Reviews and methods papers

    Reviews and methods papers

    • Highly cited

    • Influence subsequent research

    • Lay foundation

    ISI Research Services Group Nancy K. Bayers


    English language bias

    English language bias

    • English is the language of science

    • Representative journals from

      each region and nation

    • Not significant for most analyses

    ISI Research Services Group Nancy K. Bayers


    Skewed distribution in bibliometrics

    Skewed distribution in bibliometrics

    • 80/20 rule

    • 20% of papers get 80% of cites

    • 50% of papers are never cited

    ISI Research Services Group Nancy K. Bayers


    Relative measures baselines

    Relative measures--baselines

    • Need context -- what does X cites mean?

    • Normalize by size -- average citation rates

    • Normalize by field -- average field citation rates

    ISI Research Services Group Nancy K. Bayers


    What citations to not reflect

    What citations to not reflect

    • Clinical importance

    • Public policy influence

    • Regional or local interest

    ISI Research Services Group Nancy K. Bayers


    Confidence levels

    Confidence levels

    Large data sets

    (nations, regions, large institutions)

    high

    moderate

    Engineering &

    Applied Sciences

    Fundamental &

    Basic Sciences

    low

    moderate

    Small data sets

    (small departments, individual researchers)

    ISI Research Services Group Nancy K. Bayers


    In cites www in cites com

    In-Cites: www.in-cites.com

    ISI Research Services Group Nancy K. Bayers


    In cites www in cites com editorial commentary by highly cited scientists

    In-Cites: www.in-cites.comEditorial Commentary by Highly Cited Scientists

    ISI Research Services Group Nancy K. Bayers


    In cites www in cites com hot paper commentary in medicine

    In-Cites: www.in-cites.comHot Paper Commentary in Medicine

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com

    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com1

    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com2

    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com3

    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Science watch www sciencewatch com

    Science Watch: www.sciencewatch.com

    ISI Research Services Group Nancy K. Bayers


    Isi research services group1

    ISI Research Services Group:

    Thank You!!

    Analytical tools for Research Evaluation


  • Login