Archived File

1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.

2. CSR INTEGRATED REVIEW GROUP REVIEWS [peer review of peer review] Don Schneider November 3, 2008 National Institutes of HealthU.S. Department of Health and Human Services

3. CSR is committed to peer review of peer review • Need to respond to changing science • Trace roots to Panel on Scientific Boundaries for Review (PSBR) • Examine Integrated Review Group-specific issues • Discuss general issues

4. The PSBR issued a report January 2000 outlining reorganization philosophy and expectations. • URL - http://cms.csr.nih.gov/NewsandReports/ReorganizationActivitiesChannel/BackgroundUpdatesandTimeline/FinalPhase1Report.htm • Be monitored continuously: assess peer review system at regular intervals, at the level of the IRG, and globally by the Advisory Committee

5. Working Groups Assess IRGs[Peer review of peer review] • Peer reviewers report on study section operations • Feedback from reviewers condensed by staff into a brief 5-10 page white paper report • Subset of peer reviewers meet as working group to edit/confirm essence of report • Report is shared with Advisory Committee and used as basis of taking action for improvements

6. IRG Working Groups, Round 1, provided baseline, 2000-2002 • Round 1 working groups – one-cycle visit of each study section, often by single pre-selected reviewer • Pre-selection probably resulted in perturbations • Nonetheless, useful information obtained that informed reorganization • PSBR Reorganization occurred step-wise one IRG at a time from June 2003 to March 2005

7. IRG Working Groups, Round 2, were initiated in 2005 and are currently in progress • Eliminate perturbation –peer reviewers of peer review selected post-study section meetings • Solicit input from multiple reviewers each cycle • Solicit input from reviewers over multiple, at least two, cycles • Share white papers with Chairs prior to reporting to Advisory Committee • Provide substantial basis for future actions

8. Schedule for IRG Reviews July 2008 • Brain Disorders & Clinical Neuroscience (BDCN) • Molecular, Cellular & Developmental Neuroscience (MDCN) September 2008 • Integrative, Functional & Cognitive Neuroscience (IFCN) • Emerging Technologies & Training in Neuroscience (ETTN) • Biology of Development & Aging (BDA) January 2009 • Biobehavioral & Biobehavioral Processes (BBBP) • Risk, Prevention & Health Behaviors (RPHB) March 2009 • Healthcare Delivery & Methodologies (HDM) • Epidemiology & Population Sciences (EPS)

9. Cont., Schedule for IRG Reviews May 2009 • Immunology (IMM) • Infectious Diseases & Microbiology (IDM) July 2009 • Oncology 1 – Basic Translational (OBT) • Oncology 2 – Translational Clinical (OTC) September 2009 • Bioengineering Sciences & Technologies (BST) • Surgical Sciences, Biomedical Imaging & Bioengineering (SBIB) November 2009 • AIDS & Related Research (AARR) • Endocrinology, Metabolism, Nutrition & Reproductive Sciences (EMNR)

10. Cont., Schedule for IRG Reviews January 2010 • Vascular & Hematology (VH) • Cardiovascular & Respiratory Sciences (CVR) March 2010 • Musculoskeletal, Oral & Skin Sciences (MOSS) • Digestive, Kidney & Urological Systems (DKUS) May 2010 Genes, Genomes & Genetics (GGG) Biological Chemistry & Macromolecular Biophysics (BCMB) July 2010 • Interdisciplinary Molecular Sciences & Training (IMST) • Cell Biology (CB)

11. Format for IRG Reviews Late afternoon • Present Peer Review Issues & Current Initiatives (CSR Director) • Discuss; discuss further over dinner Morning • Present Working Group White papers • Describe study sections and specific issues/metrics (Chiefs) • Discuss and improve study section guidelines (Chairs) • Forecast future issues & trends in science Afternoon • Consider the critical role of the study section chair (CSR Director, in closed session with Chairs)

12. IRG specific issues and actions are noted Molecular, Cellular & Developmental Neuroscience IRG • One study section split for second time • Initial attempt to charter a neurotechnology panel led to PRAC requested crosscutting look and eventual formation of two study sections, as well as stimulating formation of crosscutting neuroscience IRG and contributing to momentum for new division

13. Cont. IRG-specific issues & actions Brain Disorders & Clinical Neuroscience IRG • Split two study sections, each into two • Resolve referral issue so that brain tumor applications focusing on neurological aspects are assigned to BDCN, with those focusing on cancer being assigned to Oncology IRG

14. Cont. IRG-specific issues & actions Integrative, Functional & Cognitive Neuroscience IRG • No issues & no actions Biology of Development & Aging IRG • Clustering of International & Cooperative Projects in BDA, with great breadth of topics – perhaps pilot editorial board review Emerging Technologies & Training in Neuroscience IRG • First applications reviewed winter 2008, no revisions/actions yet • One new study section growing rapidly, on verge of needing to split

15. General Review Issues & Actions Reviewers • Seniority • Ad hoc/temporary • Telephone • Mail • Registry/College Scoring • Streamlining/unscoring • Score compression • Scoring consistency throughout meeting

16. IRG Issues & Enhancing Peer Review Reviewers • Flexible service • Systematic training • Shorter applications and critiques Scoring • Smaller scale, 7 or perhaps 9 bins • Scores for all applications • Promote score spreading, ranking or order of review Emerging scientific trends • Selecting appropriate reviewers • Updating study section guidelines

17. The End