bibliometric analysis tools for research portfolio analysis and management n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Bibliometric Analysis Tools for Research Portfolio Analysis and Management PowerPoint Presentation
Download Presentation
Bibliometric Analysis Tools for Research Portfolio Analysis and Management

Loading in 2 Seconds...

play fullscreen
1 / 52

Bibliometric Analysis Tools for Research Portfolio Analysis and Management - PowerPoint PPT Presentation


  • 139 Views
  • Uploaded on

Bibliometric Analysis Tools for Research Portfolio Analysis and Management. Ann Kushmerick Manager, Research Evaluation and Bibliometric Data. May 3, 2013. Thomson Reuters. LARGEST PROVIDER OF INTELLIGENT INFORMATION

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Bibliometric Analysis Tools for Research Portfolio Analysis and Management' - gino


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
bibliometric analysis tools for research portfolio analysis and management

Bibliometric Analysis Tools for Research Portfolio Analysis and Management

Ann Kushmerick

Manager, Research Evaluation and Bibliometric Data

May 3, 2013

thomson reuters
Thomson Reuters
  • LARGEST PROVIDER OF INTELLIGENT INFORMATION
  • Thomson Reuters is the largest provider of intelligent information to business and professional customers in the world.

GLOBAL

OBJECTIVE

EXPERIENCED

We operate in over 90 countries around the world. We support over 5,600 universities, governments and research institutions from more than100 countries.

We offer unbiased metrics based on citation activity of the most impactful global and regional journals to the scholarly community, and remain free from proprietary involvement.

Our roots run deeper than any other information partner today. And our research evaluation experts have demonstrated experience in science policy, research administration and project management.

solutions for research funders
Solutions for Research Funders
  • Connecting research to impact for the world’s leading research funding organizations, governments, publishers, associations and institutions…

Benchmarking and comparison products

Journal evaluation and comparisons tools

  • Customized research impact evaluations
  • Expert services

Customized grant management reporting

  • Customized research portfolio analysis

RESEARCH PRODUCING ORGANIZATIONS

INTELLIGENT INFORMATION

ADVANCED TOOLS AND ANALYTICS

PROFESSIONAL SERVICES

critical decision support for research funders
Critical Decision Support for Research Funders
  • Benchmark and compare performance and output to peers
  • Identify and promote partnerships or collaborations both inside and outside the organization
  • Identify and promote areas of strength or specializations tied to impact
  • Decision support for policy and program development
  • Comprehend the research landscape
  • Provide your teams with flexible, customized systems
  • Position with board members and key stakeholders
  • Identify emerging subject areas or researchers
  • Demonstrate results and impact of research activities
  • Identify new trends and key indicators to enable policy development
web of science the foundational data for research analytics solutions
Web Of Science: The foundational data for research analytics solutions

Selectivity and control of content:consistent, transparent standards

12,000 prestigious journals and 716 million+ Cited References

Multidisciplinary: Sciences, Social Sciences, Arts & Humanities

Depth of more than 100 years, including cited references

Consistent and reliable data:

  • Cover to cover indexing
  • Cited references for all years of coverage
  • Indexing of all author names and addresses

Unmatched expertise: 50+ yrs. of citation indexing and analysis

Conferences and Scholarly Books: complete the research picture

Funding acknowledgments: link funding source to outputs

The gold standard used by 5,000+ institutions in over 100 countries

datasets to enable your analysis
Datasets to enable your analysis
  • We offer customized datasets and studies from any of our data sources:
      • Web of Science- (data can be customized or full database)
      • Medline
      • Journal Citation Reports
      • Essential Science Indicators
      • InCites
      • ScienceWire: Our data warehouse where funding inputs (grants) and outputs (journal articles, conference proceedings, patents, etc.) are linked.
about our research analytics capabilities
ABOUT OUR RESEARCH ANALYTICS CAPABILITIES

Scalable, global team of experts with strong research backgrounds

  • Highly Trained: Our global product managers, consultants, account managers and customer training teams are credentialed in all major fields and are experts in evaluation, bibliometrics, and management in Library Science, Bioinformatics, Chemistry, Neuroscience, Physics, Statistics, Mathematics, Science Policy and Evaluation.
  • Highly Experienced: We have demonstrated expertise in science policy, research administration and project management with an understanding and appreciation of the scientific discovery process. And because we have teams around the world, we are able to provide evaluations and objective reporting in global and regional contexts.
providing unmatched content tools and services
PROVIDING UNMATCHED CONTENT, TOOLS, AND SERVICES
  • Providing a clear picture of research activities and offering decision support to customers around the world - from answering a single question to creating an enterprise-wide decision support system

BENCHMARK

PROFILE

CONSULT

incites
InCites

Bibliometric analysis of research output and impact with peer comparisons.

13

questions incites answers for funders
Questions InCites answers for funders

What papers resulted from our funding? Which have the highest impact in their field?

Which of our funded authors are producing high impact research? What fields of research are they publishing in? What journals?

What institutions are we funding? How do they compare to each other?

What authors, institutions, and countries are citing our funded works?

incites offers different types of analysis to answer a variety of research evaluation questions
InCites offers different types of analysis to answer a variety of research evaluation questions
  • Research Performance Profile: Run custom reports on journal and conference literature you have funded. Measure researcher impact, collaboration success, and field strengths.

Global Comparisons: Compare research impact of countries and universities. Analyze trends back to 1981.

Institutional Profiles: Compare institutions using a 360o view. Includes bibliometric indicators, academic reputation survey, and institution submitted data,

normalized citation metrics put citation information in context
Normalized citation metrics put citation information in context
  • Citation rates vary among fields. What is good or average in mathematics is very different from what is good or average in biochemistry.
  • 23.3 cites/paper
  • H-index: 13
  • 14.5 cites/paper
  • H-index: 7
  • 9.8 cites/paper
  • H-index: 7
  • 4.2 cites/paper
  • H-index: 3
  • How “good” is this? What is the context?
  • Additional metrics are needed to understand research performance.
guidelines for citation analysis
Guidelines for citation analysis

For further guidance on citation metrics, download the white papers at: http://science.thomsonreuters.com/info/bibliometrics/http://isiwebofknowledge.com/media/pdf/UsingBibliometricsinEval_WP.pdf

  • Compare like with like – The Golden Rule
  • Use relative measures, not just absolute counts
  • More applicable to hard sciences than arts/humanities
  • Know your data parameters:
    • journal categories
    • author names
    • author addresses
    • time periods
    • document types
  • Obtain multiple measures
  • Recognize skewed nature of citation data
  • Ask whether the results are reasonable
  • Understand your data source
incites global comparisons analyze global trends in publication
InCites: Global ComparisonsAnalyze global trends in publication

Contains data on:

-countries and regions

-9 category schemes

-thousands of institutions

analyze research activity by country
Analyze Research Activity by Country

Public, Environmental, and Occupational Health is a growing field in these Asian countries, particularly China.

analyze research activity by country1
Analyze Research Activity by Country

For these countries, relative citation impact in this field is close to the world average.

1.0 represents the

world average

(field baseline)

identify which institutions are contributing to china s growth
Identify which institutions are contributing to China’s growth

These Chinese institutions are producing the most research in this field.

incites research performance profile custom built for detailed analysis of your publications
InCites: Research Performance Profile custom built for detailed analysis of your publications
options for building a research performance profile for a funder
Options for building a Research Performance Profile for a Funder

Web of Science funding acknowledgements

Medline funding acknowledgments for records shared by Medline and Web of Science

Matching data provided by the customer: Pubmed IDs, UTs (Web of Science unique identifier), bibliographic records supplied by funder

incites includes a detailed dataset built to your specifications
InCites includes a detailed dataset built to your specifications

View detailed metrics on each article from your dataset

This paper ranks in the top 0.23% of all neuroscience reviews from 2000.

Compare a citation count to journal and category norms. This paper clearly outperforms the average.

Examples taken from Mt Sinai School of Medicine sample dataset

analyze researcher performance neurosciences
Analyze researcher performance: Neurosciences

Sort by number of documents to identify the most prolific authors

These metrics are unique to InCites. They normalize citation performance by research field.

Hof, PR received over 2 times the expected rate of citation in neuroscience, and on average his papers rank in the top 18% of neuroscience papers.

generate dashboards for a variety of citation metrics e g hof pr
Generate dashboards for a variety of citation metrics, e.g. Hof, PR

Mean percentile: on average, Hof’s papers rank in the top 20% of their respective fields

Journal actual/expected ratio: Hof gets cited 1.4 times the average for the journals he publishes in

H index: 65 papers have been cited at least 65 times each

Self citation analysis: 13% of Hof’s citations are self citations. H index without self citations is 58.

identify potential new areas of excellence analyze data by research field
Identify potential new areas of excellence: analyze data by research field

Medical ethics comprises a small portion of Mt Sinai’s total output, but the category normalized impact is over 2 times the expected rate.

understand research collaborations among your funded institutions
Understand research collaborations among your funded institutions

Mt Sinai doesn’t collaborate often with McGill Univ, but the research has high impact in its field.

what is the global impact of your research
What is the global impact of your research?

Identify government bodies, global universities, and corporations who are building on your work.

understand the co funders of your research where else are your fundees getting support
Understand the co-funders of your research Where else are your fundees getting support?

Look for specific grant numbers.

View the topics funded by Dept of Veterans Affairs

Drill down to papers funded by Pfizer.

save time when creating reports efficiently support evidence based decision making
Save time when creating reports; efficiently support evidence-based decision making
  • Save your custom reports and share with other users
  • Export reports to PDF and Excel
  • Save and share your report parameters with other users
custom projects
Custom projects

Our experts can help you answer your key evaluation questions.

slide35
Electronic Scientific Portfolio Assistant (eSPA): analyzes linkages from grants to their measurable outputs

36

the global landscape of autism research
The Global Landscape of Autism Research

HHS IACC/OARC Autism Spectrum Disorder:

The Global Landscape of Autism Research

http://iacc.hhs.gov/publications-analysis/july2012/index.shtml

How does the volume of autism publications align with a strategic plan advancing autism research?

Automated classification of >10,000 publications with 85% accuracy.

the global landscape of autism research1
The Global Landscape of Autism Research

Share of ASD publications globally.

“All Other Countries” category is led by China, Taiwan, South Korea, India, Brazil, and Poland

The US accounts for the largest number of ASD publications each year

The contribution from other countries is growing over time

case study alzheimer s research uk
Case Study: Alzheimer’s Research UK
  • A UK-based biomedical charity was concerned about research capacity in dementia and wanted to use publication data to investigate the statement “research capacity in dementia is lower than in other diseases such as cancer or heart disease”

http://www.alzheimersresearchuk.org/siteFiles/resources/documents/reports/ARUK_Defeating_Dementia_-_Building_capacity_to_capitalise_on_the_UKs_research_strengths.pdf

global research report neglected tropical diseases
Global Research Report: Neglected Tropical Diseases

Download free Global Research Reports: http://sciencewatch.com/global-research-reports

real time dynamic progress report evaluation system nci itraqr1

Capture Progress Reports

Real-time, Dynamic Progress Report Evaluation System (NCI-iTRAQR)

2007

2004

2005

2006

  • Build grant progress report data model, database, and data entry tool
  • Improved data collection on progress report forms
  • Use bibliometrics to identify research areas, collaboration, and merging of fields.
  • Examine research collaborations using co-authorships and network graphs

2010

2009

2008

slide42

Patent

Tracking non-publication outcomes:How has the National Cancer Institute supported development of breast cancer biomarkers?

Indication

Biomarker

  • Integrity Biomarker data includes references to Web of Science publications and US PTO patents.
  • Patents and Publications acknowledge funding from NCI.

Integrity Biomarkers Data

Validity Status

Acknowledgement of

NCI Support

NCI Grant

Publication

slide43
Tracking non-publication outcomes: How has the National Cancer Institute supported development of FDA approved Drugs?

NCI Grant

Publication

NDA-Listed Patent

[USPTO Patent]

Approved Drugs

[FDA Orange Book]

  • FDA products linked to NCI funding through acknowledgment in patents listed in New Drug Applications (NDA).
  • Direct patent acknowledgements represent direct links to the NCI projects.
  • Additional candidate links established through non-patent references found in the patents listed in the NDA.

Non-Patent Reference List

Patent Acknowledgement of NCI Support

Acknowledgement of NCI Support

slide44

CUSTOM INDICATORS AND EVALUATIONSBreakthrough Indicator- Our goal is to create a Breakthrough Paper Indicator as a multidimensional vector

  • Multi-dimensional vector for predicting breakthroughs based on publication characteristics such as:
    • Citation rates
    • Geographical patterns of references and citations
    • Disciplinary patterns of references and citations
innovative medicines initiative joint undertaking imi evaluation
Innovative Medicines Initiative Joint Undertaking (IMI) Evaluation

The Innovative Medicines Initiative Joint Undertaking (IMI) is a public private partnership between the European Union and the European Federations of Pharmaceutical Industries and Associations (EFPIA).

With a €2 billion euro budget, IMI supports collaborative research projects and builds networks of industrial and academic experts in Europe that will boost innovation in healthcare.

The first funding call to researchers was announced in 2008

innovative medicines initiative joint undertaking imi evaluation1
Innovative Medicines Initiative Joint Undertaking (IMI) Evaluation

IMI wanted to evaluated its funding by:

  • Identifying publications
  • Citation metrics and analysis
  • Innovation and relevance to new technologies
  • Network analysis
  • Journal and patent analysis

First report (October2012):

http://www.imi.europa.eu/sites/default/files/uploads/documents/IMI%20JU%20Bibliometric%20Report%201_FINAL.pdf

Second report (March 2013): 

http://www.imi.europa.eu/sites/default/files/uploads/documents/IMI_BibliometricReport_2_final.pdf

thank you

Thank You

http://researchanalytics.thomsonreuters.com/

ann.kushmerick@thomsonreuters.com