1 / 48

Archived File

Archived File. The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files. Improving Peer Review. CSR Initiatives toni scarpa.

crwys
Download Presentation

Archived File

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.

  2. Improving Peer Review CSR Initiatives toni scarpa PRAC, April 30, 2008 National Institutes of HealthU.S. Department of Health and HumanServices

  3. Peer Review at CSR • New Data • Improving Peer Review: CSR Initiatives • Changes In CSR Organization • Realigning CSR Peer Review

  4. New Data

  5. Number of Applications Submitted Historical Growth

  6. SBIR Applications 7000 INST CSR 6000 5000 4000 3000 2000 1000 0 1999 2000 2001 2002 2003 2004 2005 2006 2007

  7. Success Rates of Competing SBIR Applications 60 SBIR Phase I SBIR Phase II 50 40 Percent Success Rate 30 20 10 0 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Fiscal Year

  8. R01 Applications Reviewed at CSR vs. Other ICs

  9. R21 Applications Reviewed at CSR vs. Other ICs

  10. % of RO1 Application Submitted, Type 1

  11. % of RO1 Application Submitted, Type 2

  12. Reviewer’s Load Applications Per Reviewer Oct Council Rounds

  13. Colleagues Reviewing for CSR

  14. CSR Budget for 2008 • CSR Budget $ 63 M • SREA (Reviewers Expenses Budget) $ 39 M • Review Cost, including travel for 17,000 reviewers, is less than 0.2% of the budget requested in the applications reviewed

  15. SREA Budget Progression Millions

  16. Travel Purchase non-refundable (restricted) coach tickets instead of unrestricted tickets

  17. Additional Savings

  18. Improving Peer Review CSR Initiatives

  19. Improving Peer Review Matrix A: Changes in CSR Operations Matrix B: Changes in Peer Review Matrix C: System Changes?

  20. Matrix A: Changes in CSR Operations • CSR Reorganization of Division and IRGs • Review Enabling Committees • Increase Efficiency • Division Directors, Chiefs and SROs Recruitment

  21. Matrix A: Changes in CSR Operations • CSR Reorganization of Division and IRGs

  22. Matrix A. Changes in CSR Operations • CSR Reorganization of Division and IRGs • Realign Divisions and IRGs with the changes in science • Increase efficiency and effectiveness • Increase consistency • Recruitment of Scientific Review Officers

  23. New Division Neuroscience, Aging and Development • BDCN/Brain Disorders & Clinical Neuroscience (est. 11 SRAs) • IFCN/Integrative, Functional, & Cognitive Neuroscience (est. 9 SRAs) • MDCN/Molecular, Cellular & Developmental Neuroscience (est. 10 SRAs) • ENST/Emerging Neuroscience & Training (est. 8 SRAs) • BDA/Biology of Development & Aging (7 SRAs)

  24. Organization of Peer Review at CSR I.R.G.s (Integrated Review Groups) Study Sections Divisions

  25. Division A (Neuroscience, Development and Aging) Division B (AIDS, Behavioral and Population Sciences) Division C (Basic and Integrative Biological Sciences) Division D (Physiological and Pathological Sciences) Division E (Translational and Clinical Sciences) Brain Disorders and Clinical Neuroscience IRG (BDCN) Biobehavioral and Behavioral Processes IRG ( BBBP) Biological Chemical and Macromolecular Biophysics IRG (BCMB) Endocrinology, Metabolism, Nutrition and Reproductive Sciences IRG (EMNR) Cardiovascular and Respiratory Sciences IRG (CVR) Molecular, Cellular and Developmental Neuroscience IRG (MDCN) Risk, Prevention and Health Behaviors IRG (RPHB) Bioengineering Sciences and Technologies IRG (BST) Immunology IRG (IMM) Surgical Sciences, Biomedical Imaging and Bioengineering IRG (SBIB) Epidemiology and Population Sciences IRG (EPS) Cell Biology IRG (CB) Infectious Diseases and Microbiology IRG (IDM) Musculoskeletal, Oral And Skin Sciences IRG (MOSS) Integrative, Functional and Cognitive Neuroscience IRG (IFCN) Genes, Genomes and Genetics IRG (GGG) Healthcare Delivery and Methodologies IRG (HDM) Digestive, Kidney and Urological Systems IRG (DKUS) Oncology 2 – Translational Clinical IRG (OTC) Emerging Technologies and Training in Neuroscience IRG (ETTN) Oncology 1 – Basic Translational IRG (OBT) AIDS and Related Research IRG (AARR) Vascular and Hematology IRG (VH) Biology of Development and Aging IRG (BDA) Interdisciplinary Molecular Sciences and Training IRG (IMST) New CSR Review Divisions Scientific Review Groups= 48 Scientific Review Groups= 44 Scientific Review Groups= 55 Scientific Review Groups= 43 Scientific Review Groups= 50

  26. Don Schneider

  27. Matrix A: Changes in CSR Operations • CSR Reorganization of Division and IRGs • Review Enabling Committees

  28. Review Enabling Committees • Policy (Martin) • Scientific Community Outreach (Schneider) • Recruitment of SROs (Gibson) • Recruitment of Reviewers (Stassi) • Knowledge Management (Many) • Peer Review Electronic (Many) • SRA Training (Freund) • Mentoring and staff development (Many) • Best review practices (Kitt)

  29. Matrix A: Changes in CSR Operations • CSR Reorganization of Division and IRGs • Review Enabling Committees • Increase Efficiency

  30. Increase Efficiency How we distribute 80,000 applications? Retooled for electronic submission • Applications are now submitted electronically Assign applications using text fingerprinting Implementation by August 2008

  31. Matrix A: Changes in CSR Operations • CSR Reorganization of Division and IRGs • Review Enabling Committees • Increase Efficiency • Division Directors, Chiefs and SROs Recruitment

  32. Matrix A: Changes in CSR Operations Recruitment 3 Division Directors First Search (Richard Nakamura, Chair) 6 Integrated Review Chiefs 40 Scientific Review Officers

  33. Matrix B: Changes in Peer Review • Improve study section alignment and performance • Shorten the review cycle • Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers • Improve the identification of significant, innovative and high-impact research

  34. CCSR Recent Activities in Peer ReviewSR Recent Activities in Peer Review • Improve Study Section Alignment and Performance

  35. Second Round of Internal IRG Review • July 24, 2008: BDCN and MDCN • September 15, 2008: ETTN, BDA and IFCN • January 2009: BBBP and HOP 2 (HDM) • March 2009: RPHB and HOP 1 (EPS) • May 2009: IMM and IDM • July 2009: ONC 1-Basic and ONC 2-Clinical • September 2009: BST and SBIB • November 2009: AARR and EMNR • January 2010: VH and CVR • March 2010: MOSS and DRUS • May 2010: GGG and BCMB • July 2010: IMST and CB

  36. Two Major Challenges • Percentiling or Ranking Applications Reviewed in SEPS • Locus of Review of “Orphan Applications”

  37. Don Schneider Clustering of grant application for review

  38. CCSR Recent Activities in Peer ReviewSR Recent Activities in Peer Review • Improve Study Section Alignment and Performance • Shorten the review cycle

  39. Shortening the Review CycleThe Goal • To provide applicants a review and score within 3 months of application submission. This will permit resubmission of applications (when doable and desirable) 4 months earlier than in the past. • To permit 3 reviews within one year

  40. - 180 160 51 140 120 100 29 30 44 80 124 60 40 74 72 60 20 0 200601 200605 200610 200701 200705 200710 Council Cycle of Original R01 New Experienced R01 A1 Resubmissions Reviewed in the Cycle Immediately following the Original Application's Review

  41. CCSR Recent Activities in Peer ReviewSR Recent Activities in Peer Review • Improve Study Section Alignment and Performance • Shorten the review cycle • Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers

  42. Colleagues Reviewing for CSR

  43. Academic Rank of All CSR Reviewers

  44. Near-Term Solutions for Recruiting and Retaining the Best Reviewers Remake peer review a learning experience • Develop a national registry of volunteer reviewers • Searchable Database with 2700 reviewers • Move a meeting/year to the west coast • 50 % this year, 90% next year • Provide tangible rewards for reviewers • No submission deadlines for permanent members of Study Sections (effective February 2008) Shorten applications • Require less travel by expanding peer review platforms

  45. Electronic Reviews • Video Enhanced Discussions • Asynchronous Electronic Discussions Expanding Peer Review’s Platforms Study Sections Necessity ●Clinical reviewers Preference ● Physicists, computational biologists New Opportunities● Fogarty, International Reviewers Goal: 15% of all reviews to be electronic in 2008

  46. Matrix C: System Changes? • The Advisory Committees of NIH Director

  47. Advisory Committees to the NIH Director • The Charge from Dr. Zerhouni: “Fund the best science, by the best scientists, with the least administrative burden…” A Self-Study by the NIH in Partnership with the Scientific Community to Strengthen Peer Review in Changing Times http://enhancing-peer-review.nih.gov

  48. This is CSR

More Related