peer review advisory committee 2 1 2010 pete morton and paul sheehy
Download
Skip this Video
Download Presentation
Peer Review Advisory Committee 2/1/2010 Pete Morton and Paul Sheehy

Loading in 2 Seconds...

play fullscreen
1 / 31

Morton Sheehy presentation PRAC 02012010 - PowerPoint PPT Presentation


  • 370 Views
  • Uploaded on

Electronic Research Administration and the Receipt and Referral of Grant Applications. Peer Review Advisory Committee 2/1/2010 Pete Morton and Paul Sheehy. Context and Drivers. Continuous evolution of Processes and Policies Interdependencies of Business Processes.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Morton Sheehy presentation PRAC 02012010' - HarrisCezar


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
peer review advisory committee 2 1 2010 pete morton and paul sheehy

Electronic Research Administration and the Receipt and Referral of Grant Applications

Peer Review Advisory Committee

2/1/2010

Pete Morton and Paul Sheehy

context and drivers
Context and Drivers
  • Continuous evolution of Processes and Policies
  • Interdependencies of Business Processes

Evolving needs of NIH’s extramural

activities:

  • IMPAC II is nearing 20 years old
  • Original System Structure Needs Refreshment
    • Lower Operations and Maintenance Costs
    • Greater Flexibility

New technologies are now available

  • IT Systems Development and Business Processes must be closely linked
  • Do not want to simply re-build the same thing without seeing if business processes would benefit from a change

Fundamental shift in the way IT is developed:

three distinct activites
Three distinct activites

IT Refreshment Strategy

Three distinct activities:

  • Develop New Hardware Infrastructure
    • Servers and SANs
    • Status: Complete
  • Develop New Software Infrastructure
    • Underlying IT applications and data tables
    • Status: Ongoing
  • Reengineer Applications “Evergreening”
    • Incorporate new infrastructure capacities
    • Refresh IT to incorporate new policies and business processes
    • Status: Starting
evergreening a business function
Evergreening a Business Function
  • General Strategy
    • Identify component business processes
    • Analyze current processes
      • Opportunity to identify and assess potential new support functions
    • Re-engineer IT systems
  • Approach
    • Create Business Process Models
      • “Current / As Is” state vs. “Future / To Be” state
    • Validate using focus groups of business subject matter experts
    • Model drives/directs IT re-engineering
  • Business Process Re-engineering (BPR)
    • Synthesis of Business Process Modeling and IT Reengineering
business process reengineering
Business Process Reengineering

Benefits

  • Modeling ensures an excellent understanding of actions and rules between business processes and supporting IT systems
    • Establishes credibility
  • Modeling provides a “refreshed examination of business processes” before reengineering the IT support for those processes
    • Better visualization of inefficiencies and pain points
    • Holistic view reduces surprises
    • If the existing model is accurate and adequate, can proceed directly to IT re-engineering
  • Synthesis allows for faster, cheaper software development
    • Better defined target
introduction bpr of csr receipt referral
Introduction: BPR of CSR Receipt & Referral
  • As part of a larger effort to examine eRA reengineering, a pilot project to BPR CSR Receipt & Referral was initiated in June 2008
  • Mandate
    • Examine and document current referral business process re: IC and IRG assignment
    • Measure process performance: efficiency and quality of assignment
    • Understand referral process’ benefit to NIH’s scientific mission
    • Understand causes of sub-optimal process performance
    • Develop heuristic future state business process designed to improve performance and quality of assignment
introduction division of receipt and referral drr
Introduction: Division of Receipt and Referral (DRR)
  • DRR fulfills four business purposes:
    • Application assignment based on the scientific content of the application
      • Technical merit review group
      • Institute or Center
    • Application quality control
    • Policy enforcement
    • Facilitate and manage changes to assignment
  • All are critical to NIH’s scientific mission
introduction business challenges
Introduction: Business Challenges
  • Increase in volume of applications
  • Requirements for consistency, flexibility, transparency, equity, accountability
  • Aggressive implementation of electronic submission
  • Sometimes problematic IMPAC II migration to web
  • Increasing complexity of assignment decisions
    • Clinical, translational and multi-disciplinary research
    • Complexity is both review (subject matter, techniques, etc) and referral (relevance to multiple IRGs and/or IC missions)
      • e.g. A developmental disorder with cardiac & behavioral manifestations
approach
Approach
  • Validate current state business process models developed by DRR and Office of the Chief IT Architect
  • Establish metrics for the process
  • Identify areas of challenge for focus
  • Perform root cause analysis on these areas
  • Review potential solutions to issues including technology and process changes
  • Develop a new business process model
methodology components
Methodology: Components

Two methodologies needed:

  • Modeling: To document current processes and heuristic future process
    • Business Genetics XBML
  • Metrics: To measure efficiency and effectiveness of current process and provide input to future state processes
    • Lean Six Sigma
methodology activities
Methodology: Activities
  • Phase 1: Update of existing model
    • 2006 OCITA current state business process model used as a foundation
    • Project team, DRR staff, IRG chiefs reviewed, corrected/updated as necessary
    • Focus on IRG assignment, not IC
  • Phase 2: Obtaining process metrics
    • DRR staff and IRG chiefs provided estimates of the normal time taken for each major business process step
    • “Normal” defined as time required for 80% of applications
results process efficiency
Results: Process Efficiency
  • 80% of Applications processed within one day
  • For this 80% of applications, metrics indicate overall efficiency is world class
    • Work/wait time distribution equals benchmarks regardless of industry
    • Wait time generated in part by policy
  • Conclusions
    • Efforts focused on reducing “wait-time” of grant applications will result in minimal improvement to referral process performance
    • Process changes should focus on the 20% of applications that take longer to process
    • However, there is still significant benefit to automating the less complex assignments
results process efficiency 2
Results: Process Efficiency (2)
  • Remaining applications much more variable in time to refer
    • 20% of total applications
  • 2/3 due to problems with application itself
    • 13% of total applications
  • 1/3 are more complex to assign
    • 7 % of total
      • Scientific overlap between IRGs
      • Complexity of application
results process efficiency of complex assignments
Results: Process Efficiency of Complex Assignments
  • Factors contributing to complex assignment:
    • Overlaps/gaps in IC mission
    • Variation in sponsorship and use of activities
    • Complexities of locus of review
    • All require scientific judgment
  • Quality of assignment
    • Essential to validity of NIH Peer Review Process
    • BP and IT changes must retain and enhance quality of assignment going forward (i.e. not bouncing back from SRG)
  • Assignment delays impact review groups
    • May delay meetings
    • May require a new special emphasis panel
system changes issues with the era system
System Changes: Issues with the eRA System
  • CSR staff discussions yielded the following issues with eRA:
    • Performance and reliability of Referral and Review modules
    • Fragility – Changes tend to introduce new problems
    • Security model does not allow others to view all referral data
    • Notes Field
      • Only place to track referral process e.g. Referral officer and IRG Chief and SRO comments
      • Lack of carryover between Receipt and Referral (RR) and Review (REV) modules
        • DRR uses RR, IRG Chiefs and SROs use REV, notes are not visible to each other
      • Lack of date/time/author stamping
      • Lack of carryover from ExitRamp
    • Release of applications and visibility to reviewers in IAR
    • Web-related issues
      • Inability to open more than one application window at a time
system changes drr priority list
System Changes: DRR Priority List

DRR staff-identified priorities:

  • Support for fully electronic Awaiting Receipt of Application (ARA) notices
  • IRG Chief push to final assignment
  • Changes, fixes and improvements in the ACR system
  • Implement single 2 day extension of the 3 day assignment window
  • Support for automated comparison of application content
      • Identification of duplicate applications and revisions
      • COTS and custom KM products to compare current and previous submissions
future model changes
Future Model: Changes
  • Suggested changes to existing processes:
    • Application Validation
      • Automate and move validations currently performed by CSR after application receipt to eSubmission process
      • Expand types of checks and validations applied to application prior to submission beyond the set limited by 424RR form set
      • Use these checks to either generate warnings for applicant or flag applications for review
      • Provide a mechanism for applications to validate their application prior to submission – “Pre-submission portal”
    • Incorporate the ability to create a special, customized workflow for specific applications that require handling outside the normal flow of operations
    • Modify the 3-day window for application release
system changes automation opportunities
System Changes: Automation Opportunities

Other changes required in order to implement the “To Be” process

  • Provide system to support validation of an application against all Grants.gov and NIH business rules prior to submission to NIH “Pre-submission portal”
  • Support for a fully structured electronic “cover letter”
    • Could be implemented in Commons without need for new OMB form
  • Enhance ARWS to production ready status
    • Fully integrate with eRA, especially notes
    • Make ARWS decisions trainable
  • Implementation of a flexible, user-controlled workflow system
  • Expand the eSubmission business rules
  • Represent Funding Opportunity Announcement as electronic business rules
  • Represent of referral guidelines as electronic business rules
suggested priorities
Suggested Priorities
  • Immediate
    • Fixes for eRA issues, especially notes
    • Support IRG Chief push to final assignment and 3 day window extension (currently planned)
    • Implement Electronic Cover Letter in Commons
      • Consider providing an optional electronic form to be submitted as an interim measure
    • Fixes for ACR system
suggested priorities cont d
Suggested Priorities – cont’d
  • Medium Term
    • Implement Pre-Submission Portal
      • Expand and harden eSubmission business rules
    • Implement ARWS at production quality
    • Implement Electronic ARAs
    • Pilot application comparison tools
  • Long Term
    • Implement Flexible Workflow
    • Implement electronic business rules for:
      • Referral Guidelines (IC assignment)
      • Funding Opportunity Announcement (application validation)
next steps era
Next Steps (eRA)

Goal: Brief review/update of desired business changes resulting from BPR then rebuild IT support for R&R incorporating desired changes

  • Form core focus group
    • Includes staff from Division of Receipt and Referral as well as CSR IRG chiefs and IC representatives
    • First meeting scheduled for mid-February
next steps era continued
Next Steps (eRA) -- Continued
  • Working with core focus group, develop detailed requirements for new R&R module
    • Design to use new eRA software infrastructure
  • Initial detailed requirements documented –mid-summer
  • Develop prototype of new module – early fall
    • Update requirements as appropriate
next steps era continued28
Next Steps (eRA) -- Continued
  • Begin iteration 1 development in fall, 2010
  • Work with focus group to test initial module and establish priorities for iteration 2 development – spring, 2011
  • Begin development of iteration 2 – summer, 2011
  • Work with focus group to test iteration 2 module and plan subsequent development if needed – late summer, 2011
  • Finalize and deploy new module into production – September 30, 2011
the next step review module
The next step: Review Module
  • As BPM is complete for the Referral module and software reengineering is starting, the next opportunity is the review module.
  • Much bigger challenge
    • peer review policy and procedures are interpreted and executed in a variety of ways
    • Operational Divisions (OpDivs) under DHHS authority utilize NIH resources but don’t necessarily follow NIH policy
    • Agency Partners utilize NIH resources but don’t necessarily follow NIH policy
  • Initial recruiting is underway
    • Governance
    • Subject matter experts
review by nih governance
Review by NIH Governance
  • Trans-NIH Senior Staff
    • Extramural Activities Working Group (EAWG)
        • Working group of NIH Steering Committee with primary responsibility for extramural research and research training
    • Information Technology Working Group (ITWG)
        • Working group of NIH Steering Committee with primary responsibility for NIH\'s enterprise-level IT programs and investments.
  • Functional Areas
    • Extramural Program Management Committee (EPMC)
        • Senior leadership of IC extramural research and research training
    • Review Policy Committee (RPC)
        • Chiefs of Review Offices
    • Review Users Group (RUG)
        • SROs with interest in electronic research administration (NIH and IC)
ad