1 / 26

Archived File

Archived File. The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files. CSR Review of Member Conflict Applications.

bijan
Download Presentation

Archived File

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.

  2. CSR Review of Member Conflict Applications Working Group Members: Samuel Edwards, SRA, CMIA Study Section, CHR Anshumali Chaudhari, SRA, VCMB Study Section Valerie Durrant, SRA, HOP-B Special Emphasis Panel Marc Rigas, SRA, BDMA Study Section

  3. Acknowledgements • Teresa Lindquist, Program Analyst Office of Planning, Evaluation and Analysis, CSR • QVR Development Team (NIH/CIT), particularly Heather Allison and Don Tiedemann • CSR IRG Chiefs • Michael Martin, Director, Division of Physiology and Pathology, CSR

  4. Overview • What is the Law and what is Policy • What is Practice • Demographics of Member SEPs • Comparison of member applications reviewed in Member SEPs and regular study sections with all CSR-reviewed applications • Conclusions and Recommendations

  5. Peer Review Regulations • Revised February 4, 2004 • 42 CFR 52h.5d: “When a peer review group meets regularly, it is assumed that a relationship among individual reviewers in the group exists and that the group as a whole may not be objective about evaluating the work of one of its members. In such a case, a member’s application or proposal shall be reviewed by another qualified review group to ensure that a competent and objective review is obtained” • Does not define what constitutes a different review group (previous version stated: no more than 50% members)

  6. CSR Policy • Member applications can be reviewed in an appropriate standing study section or special emphasis panel (SEP) • SEP Composition • Limited number (<50%) may be current or recent members of the conflicted member’s study section • Chair of SEP cannot be a member of conflicted member’s study section • SEP percentiling rules (30 – 50%)

  7. Distribution of review of member applications (combined 2005 council rounds) What is practice

  8. The proportion of member applications reviewed in regular study sections (SRG), member SEPS, and others have remained relatively constant

  9. Survey of IRG chiefs of review of member applications • Member applications are reviewed in a variety of ways within and across IRGs, mostly sister study sections and small phone SEPs. • 95% of IRGs use multiple formats to review member applications. • Appropriate expertise is the driving factor for how applications are reviewed

  10. Percent of IRGs that reported using various formats for the review of member applications (Note: Chiefs could select more than one option)

  11. Most IRGs review some or all member applications in groups (either sister study sections or SEPS) that do not contain any other current or recent members of the conflict study section

  12. MEM SEPS are held throughout the round with the peak being 1 month after the peak for regular study section meetings

  13. Most (approx. 90%) Member SEPs review 1 – 5 applications

  14. Seniority is similar for Member SEPs and standing study section meetings

  15. Overview • What is the Law and what is Policy • What is Practice • Demographics of Member SEPs • Comparison of member applications reviewed in Member SEPs and regular study sections with all CSR-reviewed applications • Conclusions and Recommendations

  16. How do study section member applications fare? • Examined only R01 applications reviewed by CSR • CSR Member SEP based on ZRG1 prefix and M suffix in the meeting code (plus Decade coding (02 – 07) beginning 2005) • Compared scores (percentiles) for member applications reviewed in Member SEPs and standing study sections (SRGs), and to all R01 applications reviewed in CSR (CSR All). • Percentage of applications in the • Top 10% • Top 20% • Bottom 70% (including streamlined applications) • Differences in Type 1 and 2 applications? • Differences for applications using human subjects?

  17. Comparison of scoring of Types 1 and 2 applications reviewed in 2005 in Member SEPs and SRG meetings and to CSR All Type 1 applications Type 2 applications Cumulative Percent Percentile Percentile

  18. 0 Type 2 applications Types 1 and 2 applications reviewed in Member SEPs and Standing Study Section meetings (SRG) for Years 2000 – 2005 compared to CSR All Type 1 applications Percentage of applications reviewed 0

  19. Fewer member applications reviewed in Member SEPs are streamlined compared to those reviewed in standing study section (SRG) meetings

  20. 0 Types 1 and 2 applications the year prior to beginning SS service compared to those reviewed in Member SEPs or study section (SRG) meetings and to CSR All Percentage of applications reviewed

  21. Comparison of applications reviewed in small SEPs (Series 90) to member SEPs and to CSR all

  22. Conclusions • Member conflict applications comprise a significant segment of applications. • Lots of members + high productivity of members = many member conflict applications • Members are providing a service to NIH • Goal is a fair review for member applications given that by regulation they cannot be reviewed in their parent study section, which may be the most appropriate study section

  23. Conclusions: Key findings No SYSTEMATIC bias in the review of members’ applications and, in particular, with the use of member SEPs. Composition (seniority) is similar The percentage of Type 2 applications that receive scores in the top 10 percentile are very similar when reviewed in member SEPS or in regular study sections Members receive comparable scores before and during SS service.  Member applications do better—likely because members are excellent scientists who write good applications.

  24. The current system provides appropriate means for reviewing member applications • IRGs differ, study sections differ, and the review of member applications will differ accordingly. • Flexibility is important to ensure most appropriate review venue for member applications. • IRG chiefs are integral!

  25. Concerns with member SEPs • Confidentiality of reviewers in small SEPs – adequate protection? • Designer reviews? • SRA workload

  26. Recent changes • Encouraged to schedule meetings earlier • Encouraged to avoid 1 or 2 application meetings

More Related