Using Metrics to Inform, Enforce and Prepare

1. Using Metrics to Inform, Enforce and Prepare Julie Cole, Duke University Todd Leovic, Duke University Panda Powell, UNCW Sarah Smith, UNCGA

2. Learning Objectives Present examples of formalized performance and monitoring programs implemented at the: System-level Campus-level Department-level Present example of how performance and monitoring results can be used to determine and train to needed competencies

3. Setting the Stage The UNC System and Duke have developed programs for monitoring performance using data – both have evolved. Commonalities include: Management support Campus/unit-level ownership Regular data collection and management review Individual campuses and departments are utilizing this information to build more compliant structures The confluence of operational data and informed training helps the institutions become even more prepared, informed and compliant

4. Informing Compliance: UNC- GA and Duke UniversitySarah M. Smith UNC-GAJulie Cole Duke

5. UNC FIT History Presidential and Board of Governors Priority Operational Risk Assessment Strengthen the control environment Improve operational efficiencies General Accounting, Financial Aid, Contracts and Grants, Student Accounts, Capital Assets Inform

6. UNC FIT Organization/Design UNC Project Management Officer Executive Steering Committee/BOG Audit Committee Campus-based Project Managers Process Leads and Advisory Teams Baseline Standards Performance Indicators Monitoring Inform

7. C&G Standards Standards ≠ Policy Dynamic and reviewed regularly Identify critical elements of the grant life cycle and best practices Document Operating Standards Account set-up Time and effort Grant Management Billings/drawdowns Reporting Closeout Inform

8. Key Performance Indicators Indicators for Each Operating Standard Establish Performance Targets Report Indicators Monthly Management Review (campus and BOG) Below Target/Negative Trending Remediation plan Inform

9. Key Performance Indicators Number of days to fund set up Number/percent of T/E reports outstanding Number/percent of late salary transfers (>90 days old) Number of A-133 certification letters outstanding Dollar/percent of accounts receivable >60, >90, >120 days Number/percent of funds with end date >120 days old and fund is still active Inform

10. Monitoring Compliance Checklist/Assessment of Internal Controls On-site Monitoring Visit Risk-based, Informed by KPI Results Document Review Report to Management Inform

11. UNC FITPerformance Funding Model Adopted FY13 10 Measures to Determine Eligibility for Funds Performance linked to enrollment growth projections UNC FIT Composite Score Audit, Internal Control Assessment, Compliance with Standards, KPIs and Monitoring Inform

12. Financial Assessment Management Senior Leadership Audit Committee for Board of Trustees (Institutional Risk) Management Centers (Management Oversight) Department (Mitigation) Inform

13. RCC Metrics: • Very similar to UNC – GA • Focus on Financial Assessment Management (FAM) • Provide useful reporting to better understand risk at all levels • Utilize sufficiently detailed analysis to fully understand underlying root causes in specific units/schools/colleges • Example: 5 projects in Period 7 were responsible for 93% of the $ in untimely NSCT moved on to federal awards • Example: 76% of all untimely transfers aggregated to 2 PIs • Example: trend analysis indicates 80% of all untimely transfers occurred in 6 departments over an annual period • Provides insight into policy and process adjustments • Example: Adjusting metrics if necessary (other examples: adjusting definitions of cost transfers to exclude parent/sub, etc.) • Provides a consistent reporting and assessment platform for all levels of the university Inform

14. Enforcing the MetricPanda Powell, UNCW

15. UNCW – Campus Perspective Compilation of data for reporting Systems used RAMSeS Banner Departments involved Sponsored Programs Controller’s office/billing Enforce

16. New fund establishment • Average number of days • Cause of “yellow” or “red” • Risk of consistent “red” section • Time & Effort Certifications • Timeliness of certifications • Banner T&E system • Training needs? • Any system issues? Enforce

17. Salary Transfer Requests • >90 days – why? • Transfers affect Time & Effort – Banner system • Risk mitigation • A-133 Audit Certifications • Critical for Sub-recipient monitoring • Review of sub-award entities • Vendor versus true Sub-award Enforce

18. Accounts Receivable • Age of receivables • Reasons for slow or non-payment • Final reports not submitted • Submitted invoice lost or misplaced • Note number/type of invoicing required • Sponsored Programs and Billing work together Enforce

19. Management Review At a glance” areas of concern GREEN – everything OK Yellow – Which areas? What causing issue? Red – closer review Why red? Is there reasonable explanation? Look for Trends Enforce

20. Todd Leovic Immunology Duke University Duke Department Use of Internal Controls Enforce

21. Internal Control Types • Operational • Promotes operational effectiveness and efficiency as well as adherence to policies and procedures. • Financial • Designed to safeguard assets and ensure completeness, accuracy and reliability of financial records. • Compliance • Ensures compliance with applicable laws and regulations. Enforce

22. Financial Assessment Management, FAM -Establishes guidelines for Cost Object activity oversight and fiscal responsibility. Including assessment, verification and substantiation for account level transactions. -FAM considers the Internal Control structure of a business unit and the preventative controls gained with the implementation of new technologies and allows flexibility based on funding source and risk level. - FAM is a process to detect corrections necessary for the accuracy, integrity and reliability of reported financial reports. Enforce

23. Internal Control Environment ( ICE ) Scope The Internal Control Environment (ICE) Assessment document focuses on the financial analysis and oversight, risk assessment, internal controls, proper training and documentation for all aspects of the department’s revenue, expenditures, and balance sheet accounts. Each ICE covers all object codes owned or managed by a unit. Enforce

24. Internal Control Environment ( ICE ) Roles and Responsibilities The individual completing the ICE document must ensure compliance with Duke Policy, GAP, and other external reporting agencies. Individuals performing FAM reviews must have appropriate knowledge of the transactions, delegated responsibility for fund oversight and proper training. Management Centers are responsible for approving the ICE document. Enforce

25. Internal Control Environment ( ICE ) Procedures Risk Assessment: Each transaction category documents the Institutional risk level. The Business Unit is expected to document the internal controls in their unit that differ from the institutional overview. The Business Unit risk assessment is used for the basis for transaction sampling. Enforce

26. Internal Control Environment ( ICE ) Procedures Sampling: Preventative controls enable responsible financial persons to use sampling quality assurance reviews. In general, higher risk will result in a larger sample size, whereas, lower risk may produce smaller sample sizes. Besides risk, key deciding factors are funding source, internal controls and historical performance. Enforce

27. Internal Control Environment ( ICE ) Procedures Transaction Review and Assessment: A transaction review and assessment for selected sample transactions should consider reasonableness and accuracy of funding for the transaction and general ledger account. Sponsored funds should also assess the allowability and allocability of the goods, services and wages/effort charged to the fund. Enforce

28. Internal Control Environment ( ICE ) Procedures Analytical Assessment: Compares actual performance to budgeted expectations and /or to projected performance, in total , by cost and general ledger line item. Analysis should identify variances requiring explanation and discussion, including overdraft status. Types of assessment include: Supporting documentation, Noting errors and corrections, Documenting Completed assessment. Enforce

29. Accounts Payable Check Request (APCR) including: Petty cash, check requests, Experimental Subject Payments and Short-Term Research Funds Institutional Risk Level: Medium ( reliance on preventative and detective controls) Institutional Internal Control Environment Overview: Accounts Payable Check Requests (APCRs) receive at least one approval signature prior to processing for payment. Accounts Payable manually inputs payment information from the hard-copy form provided. Posting errors may occur for cost object, general ledger account and payment amount fields. Recommended FAM Review Procedures: On a sample basis, compare transactions to supporting documentation. The FAM reviewer should perform analytical procedures to assess reasonableness. Example of ICE transaction category Accounts Payable Check Request (APCR) including: Petty cash, check requests, Experimental Subject Payments and Short-Term Research Funds Institutional Risk Level: Medium ( reliance on preventative and detective controls) Institutional Internal Control Environment Overview: Accounts Payable Check Requests (APCRs) receive at least one approval signature prior to processing for payment. Accounts Payable manually inputs payment information from the hard-copy form provided. Posting errors may occur for cost object, general ledger account and payment amount fields. Recommended FAM Review Procedures: On a sample basis, compare transactions to supporting documentation. The FAM reviewer should perform analytical procedures to assess reasonableness.PCR) including: Petty cash, check requests, Experimental Subject Payments and Short-Term Research Funds Enforce

30. Office Of Internal Audits Enforce

31. Business Unit FAM Review Procedures: Business Unit Internal Control Environment Overview: Document the business unit internal control environment overview if different from the Institutional overview (i.e., additional internal controls or segregation of duty issues that may result in a business unit risk level that differs from the institutional risk level above). Business unit risk level is inherently the Institutional risk level of Medium unless the business unit Internal Control Environment Assessment above indicates additional internal controls that mitigate risks further or recommended internal controls are not in place in the business unit. Click here to enter text. Office Of Internal Audits Enforce

32. Technical Advances Role assignments and tracking Procurement tool, (Buy@Duke) Travel reimbursement Effort Certification Cost Transfer workflow and approval Sampling tool Re-budgeting workflow Reporting capabilities Office Of Internal Audits Enforce

33. Outcomes Improved communication and targeted training Cost savings and efficient ordering Faster reimbursement Improved tracking of effort and cost sharing Less ticking and tying Overall better internal controls Enforce

34. A Lesson Learned Adoption of promising new technology, Buy@Duke Extensive training on use of system Focus on front-end controls Objective to reduce cost transfers, ensure allowability and improve order efficiency. Road Block at Institutional level – General guidance needed. Discovered that A/A/R assessment was being assigned to individuals who lacked training/understanding of AAR. Grant managers not able to make allocability decisions as well as those closer to the science. Uncovered a fundamental issue which needed to be addressed for improving decision making, (A/A/R) in a decentralized environment. Enabled Duke to implement directed training, and enforce with internal controls and metrics Enforce

35. Preparing the Grant Management WorkforceJulie Cole, Duke University

36. Preparing for Compliance FAM Prepare

37. New RCC Training in Support of FAM • Historic RCC Training Program Changed, based on several factors: • Functional analysis of individual’s pre and post-award duties • FAM/ICE identification of training needs • New technologies: Teaching the “Why” and not just the “How” • Reclassification of grant managers from multiple job descriptions and partial grant responsibilities (1200 individuals) to consolidated groups, with REQUIRED TRAINING CERTIFICATION • University LMS manages enrollment (Saba) • Content informedby RCC Monitoring team (Risk Assessment Metrics, Department ICE feedback, etc.) Prepare

38. Philosophy Change • Moving from “the sage on the stage” to the “guide on the side” • Hands on, practical training and experiences • Providing supervisors and practitioners with meaningful evidence of learning/knowledge/practical skills – Test results and Metrics • Supported by Multi-Media resources Prepare

39. Prepare

40. Core Competency Driven: HR Function and Metrics Driven: FAM Prepare

41. Summary …..