Isi research services group
Download
1 / 40

ISI Research Services Group: - PowerPoint PPT Presentation


  • 140 Views
  • Uploaded on

ISI Research Services Group:. Analytical tools for Research Evaluation. Outline. Research evaluation: Unique features of Web of Knowledge Journal Selection Quantitative tools Cui bono: What sectors use these analyses Citation Analysis – Issues and Caveats Free Stuff!!.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'ISI Research Services Group:' - dympna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Isi research services group

ISI Research Services Group:

Analytical tools for Research Evaluation


Outline
Outline

  • Research evaluation:

    • Unique features of Web of Knowledge

    • Journal Selection

    • Quantitative tools

  • Cui bono: What sectors use these analyses

  • Citation Analysis – Issues and Caveats

  • Free Stuff!!

ISI Research Services Group Nancy K. Bayers


The web of knowledge is
The Web of Knowledge is…

  • An integrated, web-based environment providing high quality content and the tools to access, analyze, and manage scientific and scholarly information

  • Incorporating new technologies (linking, searching, and personalization) to integrate, extend, and enhance the researchenvironment

ISI Research Services Group Nancy K. Bayers


Isi research services group

Web of Knowledgesm

E-Journal Full-Text

A Free Resource!!

Evaluated, Selected Web Content

E-Patent Full-Text

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • A. Journal Selection Process

  • “A cost effective index index must restrict its

  • coverage, as nearly as possible to only those

  • items that researchers are likely to find useful.”

  • -Eugene Garfield, Ph. D.

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • … Journal Selection

    • 1. Basic Journal Publishing Standards

    • 2. Editorial Content

    • 3. International Diversity

    • 4. Citation Analysis

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

… Journal Selection

1.) Publishing Standards

a.) Timeliness

b.) Editorial Conventions

c.) English Language Bibliographic Information

d.) Peer Review

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a.) Timeliness of Publication

  • A journal must be published according to its stated publication schedule to be considered for coverage in the ISI database.

  • ISI must receive three on-time issues in sequence.

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ... Journal Selection

  • 1. Basic Journal Publishing Standards

    • a.) Timeliness of Publication

    • b.) International Editorial Conventions

    • To optimize retrievability:

    • - Informative Journal Titles

      • - Descriptive Article Titles

      • - Full Address Information for Every Author

      • - Complete Bibliographic Information for

      • All Cited References

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a.) Timeliness of Publication

b.) International Editorial Conventions

c.) English Language Bibliographic Information

- Article Titles

- Author Names and Addresses

- Cited References

- Abstracts and Keywords

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a. Timeliness of Publication

b. International Editorial Conventions

c. English Language Bibliographic Information

d. Peer Review

Application of the peer review process is another indication of the journal’s standards, the overall quality of research presented, and the completeness of cited references.

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ... Journal Selection

  • 2.) Editorial Content

    • Do we need this new journal in the database, or is the category already well covered?

    • Does this journal offer something new?

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ... Journal Selection

  • 3.) International Diversity

    • Are the authors, editors, and editorial advisory

      • board members from around the world?

    • Does this journal reflect the global context

      • in which scientific research takes place?

ISI Research Services Group Nancy K. Bayers


Isi research services group

ISI Web of Knowledge

  • ...3. International Diversity

  • Regional Journals:

  • To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI also seeks to cover the best regional journals as well.

  • ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    • ... Journal Selection

    • 4. Citation Analysis

      • Expert use of citation data help ISI identify

      • important, influential, and useful publications.

      • ISI measuresuse by analyzing citation data.

    ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    • ... Journal Selection

    • ... 4. Citation Analysis

      • New Journals

        • Has the past work of authors and editorial

        • advisory board members received citations?

    • Established Journals

    • Impact Factor

    ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    ... Journal Selection

    ...4. Citation Analysis

    Impact Factor:

    The average number of times recent articles

    in a journal were cited in a particular year.

    Nature Genetics:

    2000 Impact factor: 30.910

    ISI Research Services Group Nancy K. Bayers


    Isi research services group

    ISI Web of Knowledge

    Journal Citation Reports (JCR) -- 2001 Science Edition

    Impact Factor Calculation

    Journal: Nature Genetics

    Impact Factor: 29.600

    Cites in 2001 to articles published in : 2000 = 6,167

    1999 = 7,538

    00+99 = 13,705

    Number of articles published in: 2000 = 253

    1999 = 210 00+99 = 463

    Calculation:

    Cites to recent articles 13,705 = 29.600

    Number of recent articles 463

    ISI Research Services Group Nancy K. Bayers


    Research evaluation
    Research Evaluation

    • Web of Knowledge: unique features

      • cited references -- footnotes

      • all author names

      • all author addresses

    • Count

      • papers -- output

      • citations -- influence or impact

    • Baselines

    ISI Research Services Group Nancy K. Bayers


    Cui bono
    Cui bono?

    • Policymakers

    • Research directors & administrators

    • Researchers

    • Publishers

    • Information specialists

    ISI Research Services Group Nancy K. Bayers


    Words to the wise
    Words to the wise...

    • Citation analysis is a fair evaluative tool in those fields where publication in the journal literature is the main vehicle of communication.

    • Need multiple indicators. Citation analysis is only a quantitative

      supplement to other measures of research evaluation,

      including peer review

    • There are always exceptions to everything.

    ISI Research Services Group Nancy K. Bayers


    Citation analysis issues and caveats
    Citation analysis: issues and caveats

    • Field publication behavior differences

    • How to handle...

      • Self cites and negative cites

      • Reviews and methods papers

      • English language bias

    • Skewed distributions

    • Relative measures/baselines

    • Confidence levels

    • What citations do not reflect

    ISI Research Services Group Nancy K. Bayers


    Field citation rates average cites 1997 2001
    Field citation rates: average cites 1997-2001

    • Cell & Developmental Biology 15.9

    • Molecular Biology 10.6

    • Biochemistry & Biophysics 9.1

    • Neuroscience 7.3

    • Chemistry & Analysis 5.9

    • Physics 4.4

    • Earth Science 3.5

    • Materials Science 2.1

    • Mathematics 1.3

    ISI Research Services Group Nancy K. Bayers


    Field publication rates size
    Field publication rates/size

    Field % papers in world

    • Clinical Medicine 24%

    • Chemistry 14%

    • Physics 12%

    • Biology & Biochemistry 8%

    • Plant & Animal Science 6%

    • Materials Science 4%

    • Neuroscience 4%

    • Geosciences 3%

    • Mathematics 2%

    • Computer Science 1%

    ISI Research Services Group Nancy K. Bayers


    Self cites and negative cites
    Self-cites and negative cites

    SELF CITES

    • Cumulative nature of individual research

    • a natural and acceptable procedure if not excessive

    • Approximately 25% in biomedicine

    • 16% in plant physiology

    • 17.5% in neurobiology

      NEGATIVE CITES

    • Only 7%

    ISI Research Services Group Nancy K. Bayers


    Reviews and methods papers
    Reviews and methods papers

    • Highly cited

    • Influence subsequent research

    • Lay foundation

    ISI Research Services Group Nancy K. Bayers


    English language bias
    English language bias

    • English is the language of science

    • Representative journals from

      each region and nation

    • Not significant for most analyses

    ISI Research Services Group Nancy K. Bayers


    Skewed distribution in bibliometrics
    Skewed distribution in bibliometrics

    • 80/20 rule

    • 20% of papers get 80% of cites

    • 50% of papers are never cited

    ISI Research Services Group Nancy K. Bayers


    Relative measures baselines
    Relative measures--baselines

    • Need context -- what does X cites mean?

    • Normalize by size -- average citation rates

    • Normalize by field -- average field citation rates

    ISI Research Services Group Nancy K. Bayers


    What citations to not reflect
    What citations to not reflect

    • Clinical importance

    • Public policy influence

    • Regional or local interest

    ISI Research Services Group Nancy K. Bayers


    Confidence levels
    Confidence levels

    Large data sets

    (nations, regions, large institutions)

    high

    moderate

    Engineering &

    Applied Sciences

    Fundamental &

    Basic Sciences

    low

    moderate

    Small data sets

    (small departments, individual researchers)

    ISI Research Services Group Nancy K. Bayers


    In cites www in cites com
    In-Cites: www.in-cites.com

    ISI Research Services Group Nancy K. Bayers


    In cites www in cites com editorial commentary by highly cited scientists
    In-Cites: www.in-cites.comEditorial Commentary by Highly Cited Scientists

    ISI Research Services Group Nancy K. Bayers


    In cites www in cites com hot paper commentary in medicine
    In-Cites: www.in-cites.comHot Paper Commentary in Medicine

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com
    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com1
    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com2
    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Special topics www esi topics com3
    Special Topics: www.esi-topics.com

    ISI Research Services Group Nancy K. Bayers


    Science watch www sciencewatch com
    Science Watch: www.sciencewatch.com

    ISI Research Services Group Nancy K. Bayers


    Isi research services group1

    ISI Research Services Group:

    Thank You!!

    Analytical tools for Research Evaluation