1 / 18

eRA Changes to the IAR System For Reviewers

This article highlights the major changes to the IAR system for reviewers, including the new 1-9 scoring system, scoring of individual core criteria, and templates for structured critiques. It provides information on the changes coming to review meetings and how they will impact the scoring process.

dhouston
Download Presentation

eRA Changes to the IAR System For Reviewers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. eRA Changes to the IAR System For Reviewers March 25, 2009 NIH Office of Extramural Research Division of Communications and Outreach

  2. Major changes Get ready! Changes coming to Review Meetings Considering Potential FY2010 funding and beyond: • New 1-9 Scoring System • Scoring of Individual Core Criteria • Templates for Structured Critiques

  3. 1-9 Scoring System; Individual Criteria • Reviewers will use a new scoring scale of 1 to 9 to list their impact/priority score • Only whole numbers will be used • One is still the best score (1=exceptional; 9=poor) • New concept: Reviewers will begin scoring the applications on five individual core criteria, using the same 9-point scale • significance, investigator(s), innovation, approach and environment (will differ for Career Development, Training, etc grant apps)

  4. Old Scoring vs. New Scoring

  5. Non-Numerical Scores • Reviewers will no longer provide non-numerical scores for preliminary scores – e.g. Not Recommended for Further Consideration (NR), Unscored (UN) - only numeric • They will continue to enter numeric and non-numeric scores for final scores • New terminology: a ‘Not Discussed’ label will replace the existing term ‘Unscored’ (UN) for individual reviewer scores and description of the application status • (Unscored no longer applies because all applications will now be given criterion scores)

  6. Structured Templates For Critiques • Reviewers will be given templates that will promote the listing of strengths and weaknesses • For each of the individual core criteria • For the overall impact • For any other relevant review consideration for any particular application, such as protection of human subjects or sharing of model organism • The templates will be provided by the Scientific Review Officer; it will initially not be available in IAR unless saved in the meeting materials folder by the Scientific Review Officer (SRO), from where reviewers can save a local copy to their computers

  7. Entering Criterion Scores • Criterion and overall scores will not be entered as part of a document (critique) upload but instead be entered on the same screen in IAR as the critique and preliminary scores • This will ensure better data integrity • Allow NIH to more accurately include the scores where needed (Summary Statements, Commons Status, etc.) • Make scores available for future analysis

  8. New: Drop Down For Five Criteria • Reviewers will see new drop-down menus in IAR for entering scores for individual criteria New drop-down

  9. Scores on Critique • Reviewers are instructed NOT to enter scores on their critique • Each reviewer’s criterion scores will be added automatically by the IAR system as a table at the beginning of each critique • Errant scores in the critiques may be removed in finalizing the summary statement

  10. Unassigned Reviewers & Discussants • At the discretion of the Scientific Review Officer: • Unassigned Reviewers may submit criterion scores and preliminary scores with or without critiques/templates (scores previously not allowed); rationale: scores from unassigned reviewers may be helpful • Discussants may be allowed to submit scores without critiques

  11. Important: No Submitting Scores Without Critique • Assigned reviewers may not submit Criterion and/or Preliminary Scores without a critique

  12. Avoiding Errors • If a reviewer tries to save the criterion and/or preliminary score without uploading the critique, he/she will receive an error message • The maximum file size for a critique is 1 MB

  13. More on Criterion Scores • Entering criterion scores is not mandatory • Reviewers may choose to enter scores only for a couple of criteria, e.g. Approach and Innovation • Criterion scores will not be averaged • Only preliminary scores from assigned reviewers will be averaged to determine a discussion cut-off point • Scores from unassigned reviewers will not be included in the preliminary score average • (pre-review meeting score)

  14. Editing Criterion Scores • Criterion scores can be updated in IAR during the submit phase, edit phase and the final scoring phase • If criterion scores are edited, the critique is regenerated each time because the critique has header information with the criterion scores. • If the criterion scores change, the critique changes

  15. Edit Criterion Scores on Voter Sheet

  16. New:Pop-up Listing Criterion Scores • The List of Applications screen will provide a link for each grant application to display the individual criterion scores in a read-only pop-up

  17. View All Scores Link to Pop-up to View All Scores

  18. Resources • Enhancing Peer Review at NIH http://enhancing-peer-review.nih.gov/index.html • Look for IAR Release Notes in the future on the eRA Web site at http://era.nih.govcommons/commons_support.cfm#iar

More Related