Peer review advisory committee 2 1 2010 pete morton and paul sheehy
Download
1 / 31

Morton Sheehy presentation PRAC 02012010 - PowerPoint PPT Presentation


  • 365 Views
  • Updated On :

Electronic Research Administration and the Receipt and Referral of Grant Applications. Peer Review Advisory Committee 2/1/2010 Pete Morton and Paul Sheehy. Context and Drivers. Continuous evolution of Processes and Policies Interdependencies of Business Processes.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Morton Sheehy presentation PRAC 02012010' - HarrisCezar


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Peer review advisory committee 2 1 2010 pete morton and paul sheehy l.jpg

Electronic Research Administration and the Receipt and Referral of Grant Applications

Peer Review Advisory Committee

2/1/2010

Pete Morton and Paul Sheehy


Context and drivers l.jpg
Context and Drivers Referral of Grant Applications

  • Continuous evolution of Processes and Policies

  • Interdependencies of Business Processes

Evolving needs of NIH’s extramural

activities:

  • IMPAC II is nearing 20 years old

  • Original System Structure Needs Refreshment

    • Lower Operations and Maintenance Costs

    • Greater Flexibility

New technologies are now available

  • IT Systems Development and Business Processes must be closely linked

  • Do not want to simply re-build the same thing without seeing if business processes would benefit from a change

Fundamental shift in the way IT is developed:


Three distinct activites l.jpg
Three distinct activites Referral of Grant Applications

IT Refreshment Strategy

Three distinct activities:

  • Develop New Hardware Infrastructure

    • Servers and SANs

    • Status: Complete

  • Develop New Software Infrastructure

    • Underlying IT applications and data tables

    • Status: Ongoing

  • Reengineer Applications “Evergreening”

    • Incorporate new infrastructure capacities

    • Refresh IT to incorporate new policies and business processes

    • Status: Starting


Evergreening a business function l.jpg
Evergreening a Business Function Referral of Grant Applications

  • General Strategy

    • Identify component business processes

    • Analyze current processes

      • Opportunity to identify and assess potential new support functions

    • Re-engineer IT systems

  • Approach

    • Create Business Process Models

      • “Current / As Is” state vs. “Future / To Be” state

    • Validate using focus groups of business subject matter experts

    • Model drives/directs IT re-engineering

  • Business Process Re-engineering (BPR)

    • Synthesis of Business Process Modeling and IT Reengineering


Business process reengineering l.jpg
Business Process Reengineering Referral of Grant Applications

Benefits

  • Modeling ensures an excellent understanding of actions and rules between business processes and supporting IT systems

    • Establishes credibility

  • Modeling provides a “refreshed examination of business processes” before reengineering the IT support for those processes

    • Better visualization of inefficiencies and pain points

    • Holistic view reduces surprises

    • If the existing model is accurate and adequate, can proceed directly to IT re-engineering

  • Synthesis allows for faster, cheaper software development

    • Better defined target


Introduction bpr of csr receipt referral l.jpg
Introduction: BPR of CSR Receipt & Referral Referral of Grant Applications

  • As part of a larger effort to examine eRA reengineering, a pilot project to BPR CSR Receipt & Referral was initiated in June 2008

  • Mandate

    • Examine and document current referral business process re: IC and IRG assignment

    • Measure process performance: efficiency and quality of assignment

    • Understand referral process’ benefit to NIH’s scientific mission

    • Understand causes of sub-optimal process performance

    • Develop heuristic future state business process designed to improve performance and quality of assignment


Evergreening era l.jpg

Business Process Reengineering the Receipt and Referral Module of eRA: a Pilot

Evergreening eRA


Introduction division of receipt and referral drr l.jpg
Introduction: Division of Receipt and Referral (DRR) Module of eRA: a Pilot

  • DRR fulfills four business purposes:

    • Application assignment based on the scientific content of the application

      • Technical merit review group

      • Institute or Center

    • Application quality control

    • Policy enforcement

    • Facilitate and manage changes to assignment

  • All are critical to NIH’s scientific mission


Introduction business challenges l.jpg
Introduction: Business Challenges Module of eRA: a Pilot

  • Increase in volume of applications

  • Requirements for consistency, flexibility, transparency, equity, accountability

  • Aggressive implementation of electronic submission

  • Sometimes problematic IMPAC II migration to web

  • Increasing complexity of assignment decisions

    • Clinical, translational and multi-disciplinary research

    • Complexity is both review (subject matter, techniques, etc) and referral (relevance to multiple IRGs and/or IC missions)

      • e.g. A developmental disorder with cardiac & behavioral manifestations


Approach l.jpg
Approach Module of eRA: a Pilot

  • Validate current state business process models developed by DRR and Office of the Chief IT Architect

  • Establish metrics for the process

  • Identify areas of challenge for focus

  • Perform root cause analysis on these areas

  • Review potential solutions to issues including technology and process changes

  • Develop a new business process model


Methodology components l.jpg
Methodology: Components Module of eRA: a Pilot

Two methodologies needed:

  • Modeling: To document current processes and heuristic future process

    • Business Genetics XBML

  • Metrics: To measure efficiency and effectiveness of current process and provide input to future state processes

    • Lean Six Sigma


Methodology activities l.jpg
Methodology: Activities Module of eRA: a Pilot

  • Phase 1: Update of existing model

    • 2006 OCITA current state business process model used as a foundation

    • Project team, DRR staff, IRG chiefs reviewed, corrected/updated as necessary

    • Focus on IRG assignment, not IC

  • Phase 2: Obtaining process metrics

    • DRR staff and IRG chiefs provided estimates of the normal time taken for each major business process step

    • “Normal” defined as time required for 80% of applications


Evergreening era13 l.jpg

Findings Module of eRA: a Pilot

Evergreening eRA


Results process efficiency l.jpg
Results: Process Efficiency Module of eRA: a Pilot

  • 80% of Applications processed within one day

  • For this 80% of applications, metrics indicate overall efficiency is world class

    • Work/wait time distribution equals benchmarks regardless of industry

    • Wait time generated in part by policy

  • Conclusions

    • Efforts focused on reducing “wait-time” of grant applications will result in minimal improvement to referral process performance

    • Process changes should focus on the 20% of applications that take longer to process

    • However, there is still significant benefit to automating the less complex assignments


Results process efficiency 2 l.jpg
Results: Process Efficiency (2) Module of eRA: a Pilot

  • Remaining applications much more variable in time to refer

    • 20% of total applications

  • 2/3 due to problems with application itself

    • 13% of total applications

  • 1/3 are more complex to assign

    • 7 % of total

      • Scientific overlap between IRGs

      • Complexity of application



Results process efficiency of complex assignments l.jpg
Results: Process Efficiency of Complex Assignments Module of eRA: a Pilot

  • Factors contributing to complex assignment:

    • Overlaps/gaps in IC mission

    • Variation in sponsorship and use of activities

    • Complexities of locus of review

    • All require scientific judgment

  • Quality of assignment

    • Essential to validity of NIH Peer Review Process

    • BP and IT changes must retain and enhance quality of assignment going forward (i.e. not bouncing back from SRG)

  • Assignment delays impact review groups

    • May delay meetings

    • May require a new special emphasis panel


System changes issues with the era system l.jpg
System Changes: Issues with the eRA System Module of eRA: a Pilot

  • CSR staff discussions yielded the following issues with eRA:

    • Performance and reliability of Referral and Review modules

    • Fragility – Changes tend to introduce new problems

    • Security model does not allow others to view all referral data

    • Notes Field

      • Only place to track referral process e.g. Referral officer and IRG Chief and SRO comments

      • Lack of carryover between Receipt and Referral (RR) and Review (REV) modules

        • DRR uses RR, IRG Chiefs and SROs use REV, notes are not visible to each other

      • Lack of date/time/author stamping

      • Lack of carryover from ExitRamp

    • Release of applications and visibility to reviewers in IAR

    • Web-related issues

      • Inability to open more than one application window at a time


System changes drr priority list l.jpg
System Changes: DRR Priority List Module of eRA: a Pilot

DRR staff-identified priorities:

  • Support for fully electronic Awaiting Receipt of Application (ARA) notices

  • IRG Chief push to final assignment

  • Changes, fixes and improvements in the ACR system

  • Implement single 2 day extension of the 3 day assignment window

  • Support for automated comparison of application content

    • Identification of duplicate applications and revisions

    • COTS and custom KM products to compare current and previous submissions


Evergreening era20 l.jpg

Recommendations Module of eRA: a Pilot

Evergreening eRA


Future model changes l.jpg
Future Model: Changes Module of eRA: a Pilot

  • Suggested changes to existing processes:

    • Application Validation

      • Automate and move validations currently performed by CSR after application receipt to eSubmission process

      • Expand types of checks and validations applied to application prior to submission beyond the set limited by 424RR form set

      • Use these checks to either generate warnings for applicant or flag applications for review

      • Provide a mechanism for applications to validate their application prior to submission – “Pre-submission portal”

    • Incorporate the ability to create a special, customized workflow for specific applications that require handling outside the normal flow of operations

    • Modify the 3-day window for application release


System changes automation opportunities l.jpg
System Changes: Automation Opportunities Module of eRA: a Pilot

Other changes required in order to implement the “To Be” process

  • Provide system to support validation of an application against all Grants.gov and NIH business rules prior to submission to NIH “Pre-submission portal”

  • Support for a fully structured electronic “cover letter”

    • Could be implemented in Commons without need for new OMB form

  • Enhance ARWS to production ready status

    • Fully integrate with eRA, especially notes

    • Make ARWS decisions trainable

  • Implementation of a flexible, user-controlled workflow system

  • Expand the eSubmission business rules

  • Represent Funding Opportunity Announcement as electronic business rules

  • Represent of referral guidelines as electronic business rules


Suggested priorities l.jpg
Suggested Priorities Module of eRA: a Pilot

  • Immediate

    • Fixes for eRA issues, especially notes

    • Support IRG Chief push to final assignment and 3 day window extension (currently planned)

    • Implement Electronic Cover Letter in Commons

      • Consider providing an optional electronic form to be submitted as an interim measure

    • Fixes for ACR system


Suggested priorities cont d l.jpg
Suggested Priorities – cont’d Module of eRA: a Pilot

  • Medium Term

    • Implement Pre-Submission Portal

      • Expand and harden eSubmission business rules

    • Implement ARWS at production quality

    • Implement Electronic ARAs

    • Pilot application comparison tools

  • Long Term

    • Implement Flexible Workflow

    • Implement electronic business rules for:

      • Referral Guidelines (IC assignment)

      • Funding Opportunity Announcement (application validation)


Evergreening era25 l.jpg

Software Reengineering Phase Module of eRA: a Pilot

Evergreening eRA


Next steps era l.jpg
Next Steps (eRA) Module of eRA: a Pilot

Goal: Brief review/update of desired business changes resulting from BPR then rebuild IT support for R&R incorporating desired changes

  • Form core focus group

    • Includes staff from Division of Receipt and Referral as well as CSR IRG chiefs and IC representatives

    • First meeting scheduled for mid-February


Next steps era continued l.jpg
Next Steps (eRA) Module of eRA: a Pilot-- Continued

  • Working with core focus group, develop detailed requirements for new R&R module

    • Design to use new eRA software infrastructure

  • Initial detailed requirements documented –mid-summer

  • Develop prototype of new module – early fall

    • Update requirements as appropriate


Next steps era continued28 l.jpg
Next Steps (eRA) Module of eRA: a Pilot-- Continued

  • Begin iteration 1 development in fall, 2010

  • Work with focus group to test initial module and establish priorities for iteration 2 development – spring, 2011

  • Begin development of iteration 2 – summer, 2011

  • Work with focus group to test iteration 2 module and plan subsequent development if needed – late summer, 2011

  • Finalize and deploy new module into production – September 30, 2011


Evergreening era29 l.jpg

Where do we go from here? Module of eRA: a Pilot

Evergreening eRA


The next step review module l.jpg
The next step: Review Module Module of eRA: a Pilot

  • As BPM is complete for the Referral module and software reengineering is starting, the next opportunity is the review module.

  • Much bigger challenge

    • peer review policy and procedures are interpreted and executed in a variety of ways

    • Operational Divisions (OpDivs) under DHHS authority utilize NIH resources but don’t necessarily follow NIH policy

    • Agency Partners utilize NIH resources but don’t necessarily follow NIH policy

  • Initial recruiting is underway

    • Governance

    • Subject matter experts


Review by nih governance l.jpg
Review by NIH Governance Module of eRA: a Pilot

  • Trans-NIH Senior Staff

    • Extramural Activities Working Group (EAWG)

      • Working group of NIH Steering Committee with primary responsibility for extramural research and research training

  • Information Technology Working Group (ITWG)

    • Working group of NIH Steering Committee with primary responsibility for NIH's enterprise-level IT programs and investments.

  • Functional Areas

    • Extramural Program Management Committee (EPMC)

      • Senior leadership of IC extramural research and research training

  • Review Policy Committee (RPC)

    • Chiefs of Review Offices

  • Review Users Group (RUG)

    • SROs with interest in electronic research administration (NIH and IC)


  • ad