Update on Peer Review

1. Cheryl A. Kitt PHD Deputy Director CSR October 30, 2009 Center for Scientific Review and National Institutes of Health U.S. Department of Health and Human Services Update on Peer Review Cheryl A. Kitt, Ph.D. Deputy Director, CSR October 30, 2009 Nationa-l Institutes of HealthU.S. Department of Health and HumanServices

2. CSR Peer Review: 2008 Statistics 77,000 applications received 16,000 reviewers 1,400 review meetings 240 Scientific Review Officers

3. CSR Peer Review: 2009 700,00 applications received Was 16,000 reviewers, now 115,000 Was 1,400 review meetings, now 38,000 Was 240 Scientific Review Officers, now 1,600

4. Enhancing Peer Review “Fund the best science, by the best scientists, with the least administrative burden…” Elias Zerhouni, MD, Former Director, NIH 2008: The Year of Peer Review

5. Amended Applications: To speed the funding of meritorious science and minimize reviewer burden: • As of January 25, 2009, all original new applications (i.e., never submitted) and competing renewal applications are permitted only a single amendment (A1).

6. Balanced and Fair Reviews Across Career Stages and Scientific Fields • New Investigator (NI): • PD/PI who has not yet competed successfully for a substantial NIH research grant • For multiple PD/PIs-all PD/PIs must meet requirements for NI status • Early Stage Investigator (ESI): • PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent) • Applies only to R01 applications • New Investigators/Early Stage Investigators are clustered together for review • TAKE HOME MESSAGE 2: New Investigators/Early Stage Investigators Apply Now!!!

7. New Investigator (NI): • PD/PI who has not yet competed successfully for a substantial NIH research grant • For multiple PD/PIs-all PD/PIs must meet requirements for NI status • Early Stage Investigator (ESI): • PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent) • Applies only to R01 applications • New Investigators/Early Stage Investigators clustered together for review

8. Enhanced Review Criteria • Overall Impact: • Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved • New Core Criteria Order: • Significance • Investigator(s) • Innovation • Approach • Environment • Review criteria enhanced and expanded and each scored from 1-9

9. Scoring To improve the transparency of the scoring process: • Applications scored on five review criteria using a scale of 1-9. • Preliminary overall impact score using 1-9 scale. • Discussed applications receive an overall impact score from each eligible (i.e., without conflicts of interest) panel member and these scores are averaged to one decimal place, and multiplied by 10. The 81 possible priority scores thus range from 10-90. • Percentiles reported in whole numbers. • All applications receive scores: • Not Discussedapplications receive initial criterion scores from the three assigned reviewers

10. Scoring Descriptions

11. What Happens at the Study Section Meeting? Discussion of applications in order of average preliminary score: Why: • Concern - variation of scores during different times of the meeting. • One recommendation was to recalibrate scores at the end of the meeting Solution: • Recalibrate dynamically throughout meeting. Requirement: • Reviewers must participate in entire meeting.

12. Critiques To improve the quality of the critiques and to focus reviewer attention on the review criteria: • Electronic template for critiques prompts for strengths and weaknesses for each criterion. • Summary statements are shorter and more focused. • Discussed applications only receive a summary of the panel’s discussion at the meeting. • ALL applications are scored. and receive critiques • Not discussed applications receive criterion scoresonly

13. Enhancing Peer Review Implementation Timeline • January 2009 Due Dates (for potential FY2010 funding) 1. Early Stage Investigator (ESI) and New Investigator Policy (NOT-OD-09-013; http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-013.html) 2. New NIH Policy on Resubmissions (NOT-OD-09-003: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-003.html) • May 2009 Review Meetings (for potential FY2010 funding) 1. 9-Point Scoring System 2. Enhanced Review Criteria 3. Formatted Reviewer Critiques 4. Scoring of Individual Review Criteria 5. Clustering of New Investigator Applications During Review • January 2010 Due Dates (for potential FY2011 funding) 1. Shorter Applications for R01s and Other Mechanisms 2. Restructured Applications to Align with Review Criteria

14. National Institutes of Health Home page from NIH web site www.nih.gov

15. Applications Reviewed by CSR (June-July 2009) • Non ARRA Applications (Normal) 16,312 • Challenges Applications 20,894 • Competitive Revisions 2,077 • GO Grants 2,697 (CSR 500?) • P 30 561 • TOTAL as of 2/6/09 40,000 plus

16. Buliding Sustainable Community-Linked Infrastructure Announcement Number RFA-OD-09-010 Title: Recovery Act Limited Competition: Building Sustainable Community-linked Infrastructure to Enable Health Science Research (RC4) Issuing Org: NIH Release Date: 09/18/2009 Receipt Date: 12/11/2009 http://grants.nih.gov/recovery

17. Historical Background of the CFS Recurring Special Emphasis Panel 103rd Congress (1993-1995) H.R.4 Title: To amend the Public Health Service Act to revise and extend the programs of the National Institutes of Health, and for other purposes. [National Institutes of Health Revitalization Act of 1993] Sponsor: Rep. Waxman, Henry A. [CA-29] (introduced 1/5/1993) S.1 Title: A bill to amend the Public Health Service Act to revise and extend the programs of the National Institutes of Health, and for other purposes. [National Institutes of Health Revitalization Act of 1993] Sponsor: Sen. Kennedy, Edward M. [MA] (introduced 1/21/1993) ================================================================== Sec.902. Chronic Fatigue Syndrome (b) EXTRAMURAL STUDY SECTION- Not later than 6 months after the date of enactment of this Act, the Secretary of Health and Human Services shall establish an extramural study section for chronic fatigue syndrome research. ================================================================== 6/10/1993 Became Public Law No: 103-43

18. TYPICAL RANGE OF EXPERTISE NEEDED FOR ZRG1 CFS PANEL MEETING Alternative Medicine Laboratory immunology Behavioral Medicine Longitudinal animal study design Biological signal transduction Longitudinal human study design Biometrics Lymphokines/cytokines Cardiovascular physiology Magnetic resonance imaging Clinical Chronic Fatigue Syndrome Neuropharmacology Clinical Fibromyalgia Neuropsychiatry Clinical immunology Neurotrophins Clinical Temporomandibular Disorders Pain control/management Cognition Pain neurophysiology Endocrinology Pain perception Epidemiology Proteomics Ethicolegal issues Psychology of chronic disease Exercise physiology/kinesiology Psychometrics Gene expression Psychoneuroimmunology Gene regulation Quality of Life assessment Genomics Receptor biochemistry and genetics Health Policy Sensory psychology Health promotion/disease prevention Sleep physiology Inflammation Sociology of chronic disease Insomnia/sleep studies

19. kittc@csr.nih.gov 301-435-1112 National Institutes of Health (NIH) U.S. Department of Health and Human Services Picture of NIH office kittc@csr.nih.gov 301-435-1112