update on peer review l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Update on Peer Review PowerPoint Presentation
Download Presentation
Update on Peer Review

Loading in 2 Seconds...

play fullscreen
1 / 19

Update on Peer Review - PowerPoint PPT Presentation


  • 203 Views
  • Uploaded on

Cheryl A. Kitt PHD Deputy Director CSR October 30, 2009 Center for Scientific Review and National Institutes of Health U.S. Department of Health and Human Services. Update on Peer Review . Cheryl A. Kitt, Ph.D. Deputy Director, CSR October 30, 2009.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Update on Peer Review' - chevelier


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
update on peer review
Cheryl A. Kitt PHD

Deputy Director CSR

October 30, 2009

Center for Scientific Review and National Institutes of Health

U.S. Department of Health and Human Services

Update on Peer Review

Cheryl A. Kitt, Ph.D.

Deputy Director, CSR

October 30, 2009

Nationa-l Institutes of HealthU.S. Department of Health and HumanServices

csr peer review 2008 statistics
CSR Peer Review: 2008 Statistics

77,000 applications received

16,000 reviewers

1,400 review meetings

240 Scientific Review Officers

csr peer review 2009
CSR Peer Review: 2009

700,00 applications received

Was 16,000 reviewers, now 115,000

Was 1,400 review meetings, now 38,000

Was 240 Scientific Review Officers, now 1,600

2008 the year of peer review
Enhancing Peer Review

“Fund the best science, by the best scientists, with the least administrative burden…”

Elias Zerhouni, MD, Former Director, NIH

2008: The Year of Peer Review
amended applications
Amended Applications:

To speed the funding of meritorious science and minimize reviewer burden:

  • As of January 25, 2009, all original new applications (i.e., never submitted) and competing renewal applications are permitted only a single amendment (A1).
balanced and fair reviews across career stages and scientific fields
Balanced and Fair Reviews Across Career Stages and Scientific Fields
  • New Investigator (NI):
    • PD/PI who has not yet competed successfully for a substantial NIH research grant
      • For multiple PD/PIs-all PD/PIs must meet requirements for NI status
  • Early Stage Investigator (ESI):
    • PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)
  • Applies only to R01 applications
  • New Investigators/Early Stage Investigators are clustered together for review
  • TAKE HOME MESSAGE 2: New Investigators/Early Stage Investigators Apply Now!!!
slide7
New Investigator (NI):
    • PD/PI who has not yet competed successfully for a substantial NIH research grant
      • For multiple PD/PIs-all PD/PIs must meet requirements for NI status
  • Early Stage Investigator (ESI):
    • PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)
  • Applies only to R01 applications
  • New Investigators/Early Stage Investigators clustered together for review
enhanced review criteria
Enhanced Review Criteria
  • Overall Impact:
    • Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved
  • New Core Criteria Order:
    • Significance
    • Investigator(s)
    • Innovation
    • Approach
    • Environment
        • Review criteria enhanced and expanded and each scored from 1-9
scoring
Scoring

To improve the transparency of the scoring process:

  • Applications scored on five review criteria using a scale of 1-9.
  • Preliminary overall impact score using 1-9 scale.
  • Discussed applications receive an overall impact score from each eligible (i.e., without conflicts of interest) panel member and these scores are averaged to one decimal place, and multiplied by 10. The 81 possible priority scores thus range from 10-90.
  • Percentiles reported in whole numbers.
  • All applications receive scores:
    • Not Discussedapplications receive initial criterion scores from the three assigned reviewers
what happens at the study section meeting
What Happens at the Study Section Meeting?

Discussion of applications in order of average preliminary score:

Why:

  • Concern - variation of scores during different times of the meeting.
    • One recommendation was to recalibrate scores at the end of the meeting

Solution:

  • Recalibrate dynamically throughout meeting.

Requirement:

    • Reviewers must participate in entire meeting.
critiques
Critiques

To improve the quality of the critiques and to focus reviewer attention on the review criteria:

  • Electronic template for critiques prompts for strengths and weaknesses for each criterion.
  • Summary statements are shorter and more focused.
  • Discussed applications only receive a summary of the panel’s discussion at the meeting.
  • ALL applications are scored. and receive critiques
    • Not discussed applications receive criterion scoresonly
enhancing peer review implementation timeline
Enhancing Peer Review Implementation Timeline
  • January 2009 Due Dates (for potential FY2010 funding)

1. Early Stage Investigator (ESI) and New Investigator Policy (NOT-OD-09-013; http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-013.html)

2. New NIH Policy on Resubmissions (NOT-OD-09-003: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-003.html)

  • May 2009 Review Meetings (for potential FY2010 funding)

1. 9-Point Scoring System

2. Enhanced Review Criteria

3. Formatted Reviewer Critiques

4. Scoring of Individual Review Criteria

5. Clustering of New Investigator Applications During Review

  • January 2010 Due Dates (for potential FY2011 funding)

1. Shorter Applications for R01s and Other Mechanisms

2. Restructured Applications to Align with Review Criteria

national institutes of health
National Institutes of Health

Home page from NIH web site www.nih.gov

applications reviewed by csr june july 2009
Applications Reviewed by CSR (June-July 2009)
  • Non ARRA Applications (Normal) 16,312
  • Challenges Applications 20,894
  • Competitive Revisions 2,077
  • GO Grants 2,697 (CSR 500?)
  • P 30 561
  • TOTAL as of 2/6/09 40,000 plus
http grants nih gov recovery
Buliding Sustainable Community-Linked Infrastructure Announcement Number RFA-OD-09-010

Title: Recovery Act Limited Competition: Building Sustainable Community-linked Infrastructure to Enable Health Science Research (RC4)

Issuing Org: NIH

Release Date: 09/18/2009

Receipt Date: 12/11/2009

http://grants.nih.gov/recovery
historical background of the cfs recurring special emphasis panel
Historical Background of the CFS Recurring Special Emphasis Panel

103rd Congress (1993-1995)

H.R.4

Title: To amend the Public Health Service Act to revise and extend the programs

of the National Institutes of Health, and for other purposes. [National

Institutes of Health Revitalization Act of 1993]

Sponsor: Rep. Waxman, Henry A. [CA-29] (introduced 1/5/1993)

S.1

Title: A bill to amend the Public Health Service Act to revise and extend the

programs of the National Institutes of Health, and for other purposes.

[National Institutes of Health Revitalization Act of 1993]

Sponsor: Sen. Kennedy, Edward M. [MA] (introduced 1/21/1993)

==================================================================

Sec.902. Chronic Fatigue Syndrome

(b) EXTRAMURAL STUDY SECTION- Not later than 6 months after the date of

enactment of this Act, the Secretary of Health and Human Services shall

establish an extramural study section for chronic fatigue syndrome research.

==================================================================

6/10/1993 Became Public Law No: 103-43

typical range of expertise needed for zrg1 cfs panel meeting
TYPICAL RANGE OF EXPERTISE NEEDED FOR ZRG1 CFS PANEL MEETING

Alternative Medicine Laboratory immunology

Behavioral Medicine Longitudinal animal study design

Biological signal transduction Longitudinal human study design

Biometrics Lymphokines/cytokines

Cardiovascular physiology Magnetic resonance imaging

Clinical Chronic Fatigue Syndrome Neuropharmacology

Clinical Fibromyalgia Neuropsychiatry

Clinical immunology Neurotrophins

Clinical Temporomandibular Disorders Pain control/management

Cognition Pain neurophysiology

Endocrinology Pain perception

Epidemiology Proteomics

Ethicolegal issues Psychology of chronic disease

Exercise physiology/kinesiology Psychometrics

Gene expression Psychoneuroimmunology

Gene regulation Quality of Life assessment

Genomics Receptor biochemistry and genetics

Health Policy Sensory psychology

Health promotion/disease prevention Sleep physiology

Inflammation Sociology of chronic disease

Insomnia/sleep studies

kittc@csr nih gov
kittc@csr.nih.gov

301-435-1112

National Institutes of Health (NIH)

U.S. Department of Health and Human Services

Picture of NIH office

kittc@csr.nih.gov

301-435-1112