Archived File

1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.

2. CSR: New Challenges and Opportunities Toni Scarpa Center for Scientific Review National Institutes of Health Department of Health and Human Services PRAC, May 2006, 2006

3. CSR Welcomes-- Deputy Director Director of Planning, Analysis, and Evaluation Cheryl Kitt Cheryl Oros Director, Extramural Research, NIAMS Director, Office of Planning and Accountability USDA extramural research program

4. CSR Progress Report • Changes that have Occurred • Changes in Progress • Changes Under Discussion • Challenges: Present and Futures

5. Necessary Changes in CSR Peer Review Operations CSR Operations Current Systems New Systems? Complexity and Impact Time

6. Changes in CSR Operations Increased Communication and Transparency Within CSR With NIH and other Agencies With the Scientific Community Increase uniformity Summary Statements Posting all within one months of Study Section Posting Summary Statements of new investigators within one week More complete and structured resumes Unscoring Unscoring 50% Increase Efficiency Electronic Submission Text Fingerprinting, Artificial Intelligence Software

7. Knowledge Management Tools for Peer Review Collexis Software or Others • Knowledge management solutions • Fingerprinting and text retrieving Benefits for Peer Review • Assigning applications to Integrated Review Groups or Study Sections • Selecting reviewers (one application, multiple applications) Major Pilot to Assign Study Sections directly by October Plan to be in Operation by February

8. Monitoring IRGs and Study Sections • Every IRG is been assessed by all senior CSR staff every 2 years in addition to the current 5-year assessments involving all stakeholders. • Emergent problems addressed by working groups with leaders from the extramural community, and senior NIH/CSR staff. • Substantive issues/changes reviewed by the NIH Peer Review Advisory Committee. • Most study sections visited by CSR’s Director and senior staff at least once a year. • All retiring study section chairs called by CSR’s Director to learn about problems and possible improvements. • All summary statements read by CSR’s Director. • Multiple outreach efforts now gather fresh input.

9. IRG Review Schedule

10. Monitoring IRGs and Study Sections • Every IRG is been assessed by all senior CSR staff every 2 years in addition to the current 5-year assessments involving all stakeholders. • Emergent problems addressed by working groups with leaders from the extramural community, and senior NIH/CSR staff. • Substantive issues/changes reviewed by the NIH Peer Review Advisory Committee. • Most study sections visited by CSR’s Director and senior staff at least once a year. • All retiring study section chairs called to learn about problems and possible improvements. • All summary statements read. • Multiple outreach efforts now gather fresh input. • Open house meetings

11. Broad Scientific Areas • Biomolecular (4): Biological Chemistry and Macromolecular Biophysics (BCMB); Biology of Development and Aging (BDA); Bioengineering Sciences and Technologies (BST); Cell Biology (CB) • Control/Regulation (4): Genes, Genomes and Genetics (GGG); Immunology (IMM); Oncological Sciences (ONC); Surgical Sciences, Biomedical Imaging and Bioengineering (SBIB) • Integrated Biological (5): Cardiovascular Sciences (CVS); Digestive Sciences (DIG); Musculoskeletal, Oral and Skin Sciences (MOSS); Respiratory Sciences (RES); Renal and Urological Sciences (RUS) • Disease-based (5): AIDS and Related Research (AARR); Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR); Hematology (HEME); Infectious Diseases and Microbiology (IDM) • Neurological (3): Brain Disorders and Clinical Neuroscience (BDCN); Integrative, Functional and Cognitive Neuroscience (IFCN); Molecular, Cellular and Developmental Neuroscience (MDCN) • Behavioral/Social (3): Biobehavioral and Behavioral Processes (BBBP); Health of the Population (HOP); Risk Prevention and Health Behavior (RPHB)

12. Possible Changes in Current Systems • Shorten the review cycle • Address concern that clinical research is not properly evaluated • Improve the assessment of innovative, high- risk/high-reward research • Do more to recruit and retain more high-quality reviewers

13. Changes in CSR Review • Shorten the review cycle

14. Shortening the NIH Review Cycle, Initial Steps We are conducting a pilot study to speed the review process for new investigators so they may revise and resubmit for the very next review cycle, 4 months earlier than before (effective Feb ‘06). We are posting Summary Statements within one month after the Study Section meeting, instead of two to three months after the meeting (effective Oct 05)

15. Changes in CSR Review • Shorten the review cycle • Address concern that clinical research is not properly evaluated

16. 22.07 17.56 M. Martin, CSR/NIH/DHHS

17. M. Martin, CSR/NIH/DHHS

18. Distribution Among Different Types of R01 Applications: All HS+ or HS- Applications – 2000-2004 M. Martin, CSR/NIH/DHHS

19. Some Observations • On a percent basis, HS+ PIs are more unlikely to apply for a type 2 A0 or A1 application • Funded type 1 new HS+ PIs that do not submit a type 2 are less likely to submit for another activity than HS- funded PIs • NIH is loosing successful, funded, HS+ PIs at a greater rate than HS- PIs [40% vs. 30%]

20. Desirable Changes in CSR Review • Shorten the review cycle • Address concern that clinical research is not properly evaluated • Improve the assessment of innovative, high- risk/high-reward research

21. Possible Changes in Current Systems • Shorten the review cycle • Address concern that clinical research is not properly evaluated • Improve the assessment of innovative, high- risk/high-reward research • Do more to recruit and retain more high-quality reviewers

22. CSR Applications Reviewed, Regular and SEP May Council Only

23. Number of Research Grant Applications/Applicant

24. Average Number of Applications Per ReviewerOctober Council Only

25. Expanding Peer Review’s Platforms Study Sections Electronic Reviews • Telephone Enhanced Discussions • Video Enhanced Discussions • Asynchronous Electronic Discussions Necessity ●Clinical reviewers Preference● Physicists, computational biologists New Opportunities●Fogarty, International Reviewers 10% of all reviews by the end of the year.

26. Challenges to Recruiting Reviewers • Decrease the number of reviewers and increase the quality • Increase the number of applications reviewed without extra workload • Recruit and retain the best reviewers

27. Possible Short Term Approaches for Increasing Efficiency for Reviewers and CSR • Replace Many SEPs with Smaller Parallel Study Sections • Enlarge Study Section Membership and Decrease Frequency of Participation • Convene Pre-meetings to Streamline • Use Various Review Platforms • Use Hybrid Review Platforms • Unscore 40% of the F32s (postdoctoral fellowships) • Shorten Applications • Create More Structured Applications and Reviews

28. Size of Grant Applications • RO1 • Will increase number of applications reviewed by reviewers • Will decrease the number of reviewers in a study section • May be combined with a change in format of the application, more consonant to review criteria • Strong support by councils and scientific leadership • May result in more innovation

29. Trans-NIH Committee to Shorten the Application Chairs Robert Finkelstein, Ph.D. (Co-Chair), NINDS Donald Schneider, Ph.D. (Co-Chair), CSR Members Mary Custer, Ph.D. CSR Ann A. Hagan, Ph.D. NIGMS Craig A. Jordan, Ph.D. NIDCD Sherry Mills, M.D., M.P.H. OER Philip F. Smith, Ph.D. NIDDK Betsy Wilder, Ph.D. NIDDK’ Alan L. Willard, Ph.D. NINDS

30. This is CSR