isi research services group n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
ISI Research Services Group: PowerPoint Presentation
Download Presentation
ISI Research Services Group:

Loading in 2 Seconds...

play fullscreen
1 / 40

ISI Research Services Group: - PowerPoint PPT Presentation


  • 145 Views
  • Uploaded on

ISI Research Services Group:. Analytical tools for Research Evaluation. Outline. Research evaluation: Unique features of Web of Knowledge Journal Selection Quantitative tools Cui bono: What sectors use these analyses Citation Analysis – Issues and Caveats Free Stuff!!.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'ISI Research Services Group:' - dympna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
isi research services group

ISI Research Services Group:

Analytical tools for Research Evaluation

outline
Outline
  • Research evaluation:
    • Unique features of Web of Knowledge
    • Journal Selection
    • Quantitative tools
  • Cui bono: What sectors use these analyses
  • Citation Analysis – Issues and Caveats
  • Free Stuff!!

ISI Research Services Group Nancy K. Bayers

the web of knowledge is
The Web of Knowledge is…
  • An integrated, web-based environment providing high quality content and the tools to access, analyze, and manage scientific and scholarly information
  • Incorporating new technologies (linking, searching, and personalization) to integrate, extend, and enhance the researchenvironment

ISI Research Services Group Nancy K. Bayers

slide4

Web of Knowledgesm

E-Journal Full-Text

A Free Resource!!

Evaluated, Selected Web Content

E-Patent Full-Text

ISI Research Services Group Nancy K. Bayers

slide5

ISI Web of Knowledge

  • A. Journal Selection Process
  • “A cost effective index index must restrict its
  • coverage, as nearly as possible to only those
  • items that researchers are likely to find useful.”
  • -Eugene Garfield, Ph. D.

ISI Research Services Group Nancy K. Bayers

slide6

ISI Web of Knowledge

  • … Journal Selection
    • 1. Basic Journal Publishing Standards
    • 2. Editorial Content
    • 3. International Diversity
    • 4. Citation Analysis

ISI Research Services Group Nancy K. Bayers

slide7

ISI Web of Knowledge

… Journal Selection

1.) Publishing Standards

a.) Timeliness

b.) Editorial Conventions

c.) English Language Bibliographic Information

d.) Peer Review

ISI Research Services Group Nancy K. Bayers

slide8

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a.) Timeliness of Publication

  • A journal must be published according to its stated publication schedule to be considered for coverage in the ISI database.
  • ISI must receive three on-time issues in sequence.

ISI Research Services Group Nancy K. Bayers

slide9

ISI Web of Knowledge

  • ... Journal Selection
  • 1. Basic Journal Publishing Standards
    • a.) Timeliness of Publication
    • b.) International Editorial Conventions
    • To optimize retrievability:
    • - Informative Journal Titles
          • - Descriptive Article Titles
          • - Full Address Information for Every Author
          • - Complete Bibliographic Information for
          • All Cited References

ISI Research Services Group Nancy K. Bayers

slide10

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a.) Timeliness of Publication

b.) International Editorial Conventions

c.) English Language Bibliographic Information

- Article Titles

- Author Names and Addresses

- Cited References

- Abstracts and Keywords

ISI Research Services Group Nancy K. Bayers

slide11

ISI Web of Knowledge

... Journal Selection

1.) Basic Journal Publishing Standards

a. Timeliness of Publication

b. International Editorial Conventions

c. English Language Bibliographic Information

d. Peer Review

Application of the peer review process is another indication of the journal’s standards, the overall quality of research presented, and the completeness of cited references.

ISI Research Services Group Nancy K. Bayers

slide12

ISI Web of Knowledge

  • ... Journal Selection
  • 2.) Editorial Content
    • Do we need this new journal in the database, or is the category already well covered?
    • Does this journal offer something new?

ISI Research Services Group Nancy K. Bayers

slide13

ISI Web of Knowledge

  • ... Journal Selection
  • 3.) International Diversity
    • Are the authors, editors, and editorial advisory
      • board members from around the world?
    • Does this journal reflect the global context
      • in which scientific research takes place?

ISI Research Services Group Nancy K. Bayers

slide14

ISI Web of Knowledge

    • ...3. International Diversity
    • Regional Journals:
  • To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI also seeks to cover the best regional journals as well.

ISI Research Services Group Nancy K. Bayers

slide15

ISI Web of Knowledge

  • ... Journal Selection
  • 4. Citation Analysis
      • Expert use of citation data help ISI identify
      • important, influential, and useful publications.
      • ISI measuresuse by analyzing citation data.

ISI Research Services Group Nancy K. Bayers

slide16

ISI Web of Knowledge

  • ... Journal Selection
  • ... 4. Citation Analysis
      • New Journals
          • Has the past work of authors and editorial
          • advisory board members received citations?
      • Established Journals
      • Impact Factor

ISI Research Services Group Nancy K. Bayers

slide17

ISI Web of Knowledge

... Journal Selection

...4. Citation Analysis

Impact Factor:

The average number of times recent articles

in a journal were cited in a particular year.

Nature Genetics:

2000 Impact factor: 30.910

ISI Research Services Group Nancy K. Bayers

slide18

ISI Web of Knowledge

Journal Citation Reports (JCR) -- 2001 Science Edition

Impact Factor Calculation

Journal: Nature Genetics

Impact Factor: 29.600

Cites in 2001 to articles published in : 2000 = 6,167

1999 = 7,538

00+99 = 13,705

Number of articles published in: 2000 = 253

1999 = 210 00+99 = 463

Calculation:

Cites to recent articles 13,705 = 29.600

Number of recent articles 463

ISI Research Services Group Nancy K. Bayers

research evaluation
Research Evaluation
  • Web of Knowledge: unique features
    • cited references -- footnotes
    • all author names
    • all author addresses
  • Count
    • papers -- output
    • citations -- influence or impact
  • Baselines

ISI Research Services Group Nancy K. Bayers

cui bono
Cui bono?
  • Policymakers
  • Research directors & administrators
  • Researchers
  • Publishers
  • Information specialists

ISI Research Services Group Nancy K. Bayers

words to the wise
Words to the wise...
  • Citation analysis is a fair evaluative tool in those fields where publication in the journal literature is the main vehicle of communication.
  • Need multiple indicators. Citation analysis is only a quantitative

supplement to other measures of research evaluation,

including peer review

  • There are always exceptions to everything.

ISI Research Services Group Nancy K. Bayers

citation analysis issues and caveats
Citation analysis: issues and caveats
  • Field publication behavior differences
  • How to handle...
    • Self cites and negative cites
    • Reviews and methods papers
    • English language bias
  • Skewed distributions
  • Relative measures/baselines
  • Confidence levels
  • What citations do not reflect

ISI Research Services Group Nancy K. Bayers

field citation rates average cites 1997 2001
Field citation rates: average cites 1997-2001
  • Cell & Developmental Biology 15.9
  • Molecular Biology 10.6
  • Biochemistry & Biophysics 9.1
  • Neuroscience 7.3
  • Chemistry & Analysis 5.9
  • Physics 4.4
  • Earth Science 3.5
  • Materials Science 2.1
  • Mathematics 1.3

ISI Research Services Group Nancy K. Bayers

field publication rates size
Field publication rates/size

Field % papers in world

  • Clinical Medicine 24%
  • Chemistry 14%
  • Physics 12%
  • Biology & Biochemistry 8%
  • Plant & Animal Science 6%
  • Materials Science 4%
  • Neuroscience 4%
  • Geosciences 3%
  • Mathematics 2%
  • Computer Science 1%

ISI Research Services Group Nancy K. Bayers

self cites and negative cites
Self-cites and negative cites

SELF CITES

  • Cumulative nature of individual research
  • a natural and acceptable procedure if not excessive
  • Approximately 25% in biomedicine
  • 16% in plant physiology
  • 17.5% in neurobiology

NEGATIVE CITES

  • Only 7%

ISI Research Services Group Nancy K. Bayers

reviews and methods papers
Reviews and methods papers
  • Highly cited
  • Influence subsequent research
  • Lay foundation

ISI Research Services Group Nancy K. Bayers

english language bias
English language bias
  • English is the language of science
  • Representative journals from

each region and nation

  • Not significant for most analyses

ISI Research Services Group Nancy K. Bayers

skewed distribution in bibliometrics
Skewed distribution in bibliometrics
  • 80/20 rule
  • 20% of papers get 80% of cites
  • 50% of papers are never cited

ISI Research Services Group Nancy K. Bayers

relative measures baselines
Relative measures--baselines
  • Need context -- what does X cites mean?
  • Normalize by size -- average citation rates
  • Normalize by field -- average field citation rates

ISI Research Services Group Nancy K. Bayers

what citations to not reflect
What citations to not reflect
  • Clinical importance
  • Public policy influence
  • Regional or local interest

ISI Research Services Group Nancy K. Bayers

confidence levels
Confidence levels

Large data sets

(nations, regions, large institutions)

high

moderate

Engineering &

Applied Sciences

Fundamental &

Basic Sciences

low

moderate

Small data sets

(small departments, individual researchers)

ISI Research Services Group Nancy K. Bayers

in cites www in cites com
In-Cites: www.in-cites.com

ISI Research Services Group Nancy K. Bayers

in cites www in cites com editorial commentary by highly cited scientists
In-Cites: www.in-cites.comEditorial Commentary by Highly Cited Scientists

ISI Research Services Group Nancy K. Bayers

in cites www in cites com hot paper commentary in medicine
In-Cites: www.in-cites.comHot Paper Commentary in Medicine

ISI Research Services Group Nancy K. Bayers

special topics www esi topics com
Special Topics: www.esi-topics.com

ISI Research Services Group Nancy K. Bayers

special topics www esi topics com1
Special Topics: www.esi-topics.com

ISI Research Services Group Nancy K. Bayers

special topics www esi topics com2
Special Topics: www.esi-topics.com

ISI Research Services Group Nancy K. Bayers

special topics www esi topics com3
Special Topics: www.esi-topics.com

ISI Research Services Group Nancy K. Bayers

science watch www sciencewatch com
Science Watch: www.sciencewatch.com

ISI Research Services Group Nancy K. Bayers

isi research services group1

ISI Research Services Group:

Thank You!!

Analytical tools for Research Evaluation