Procurement Forum March 12, 2008 Agenda

1. Procurement Forum March 12, 2008Agenda Welcome Performance Appraisal Rating System and Guidelines 210.S Overview Contractor Performance Evaluations (NF 1680) Closeout Best Practices Wrap-Up

2. Procurement Training Forum Contractor Performance Information (NF 1680)

3. OUTLINE • Policy and Procedures • Evaluation Form • NF 1680 • Rating Definitions • Best Practices • Samples

4. POLICY • FAR 42.15, NFS 1842.15, and PIC 01-12 • Purpose • Communications with Contractor • Source Selection Information • Evaluations used to determine award fee or incentive fee payments may be used to satisfy FAR requirements • Update to PPDB each award fee period for award and incentive fee contracts

5. PROCEDURES • Evaluations required for: • Contracts exceeding simplified acquisition threshold • Contracts exceeding 1 year period of performance • Purchase Orders and BOA orders over $100K • Evaluation not required for Interagency Agreements

6. PROCEDURES • Interim evaluations, annually • Within 60 days of anniversary • Optional on SBIR/STTR Phase II’s • Final evaluations at completion • Final, summarizing all performance, must be done • Minimum of 30 days for contractor comments • Review of timely disagreement (upon contractor request) at one level above Contracting Officer and must be within 30 days • Copy of all finalized evaluations must be provided to contractor

7. PROCEDURES • Copies of full evaluation (with contractor comments, etc.) must be retained and marked “Source Selection Information” • Disclosure restricted to Government personnel and the contractor during period it’s used as SSI • Retain for purposes of source selection for 3 years after completion of contract performance • Retain in contract file as provided in FAR 4.8 • Create and save NF 1680 in PPDB

8. NF 1680 • NASA Form 1680, entitled, "Evaluation of Performance," shall be used to document evaluations. • Contractor performance rated for the following: • Attributes • Quality • Timeliness • Price/Cost • Other (as specified) • Ratings • Excellent • Very Good • Good • Satisfactory • Poor/Unsatisfactory • Attributes should • be defined as part • of the evaluation • Rating definitions at NFS 1816.405-275, Award Fee Evaluation Scoring

9. RATING DEFINITIONS • Excellent: Of exceptional merit; exemplary performance in a timely, efficient, and economical manner; very minor (if any) deficiencies with no adverse effect on overall performance. • Very good: Very effective performance, fully responsive to contract requirements; contract requirements accomplished in a timely, efficient, and economical manner for the most part; only minor deficiencies. • Good: Effective performance; fully responsive to contract requirements; reportable deficiencies, but with little identifiable effect on overall performance. • Satisfactory: Meets or slightly exceeds minimum acceptable standards; adequate results; reportable deficiencies with identifiable, but not substantial, effects on overall performance. • Poor/Unsatisfactory: Does not meet minimum acceptable standards in one or more areas; remedial action required in one or more areas; deficiencies in one or more areas which adversely affect overall performance.

10. BEST PRACTICES • Start early with COTR request • Minimum of 30 days before anniversary date • Use MM or Outlook schedule reminders • Meet with COTR in-person, if possible • Give purpose for evaluation • Provide rating definitions and samples • Work with COTR and Other Stakeholders (COTR Management, Resources, QA, End Users, etc.) • Frequent follow-ups may be necessary • Use detailed and thoughtful narratives, that: • Define attributes • Assign ratings in context of rating definition • Use facts (as opposed to opinions) • Maintain consistency with assigned rating definition See Samples

11. BEST PRACTICES • Prepare and edit narratives in MS Word and cut and paste into PPDB • Provide contractor advance notice it will receive evaluation • Explain purpose of evaluation • With evaluation, provide contractor: • Rating definitions • Notice that evaluation will be placed in database after 30 days • Invitation to discuss evaluation, if appropriate • Invite COTR to evaluation discussion • Prepare COTR before discussion

12. BEST PRACTICES • Contractors may ask why they did not receive a higher rating • Discussion is not a negotiation • Ratings should not change simply because contractor disagrees • Narratives should not change, unless factually incorrect • Review rating definitions with contractor • Consistency between narrative and rating definition helps • Remind contractor that written comments on evaluation are welcome • Contractor’s comments should be added verbatim

13. BEST PRACTICES • Give COTR (and other stakeholders) a due date for responses to contractor comments • COTR will likely respond quickly, but in some cases follow-ups may be necessary • If COTR needs to coordinate with his/her manager, you may want to go to the manager (e.g., Project Manager) • NASA review should -- • Address contractor’s disagreements • Document discussions, if any, held after comments received • Summarize final findings

14. BEST PRACTICES • Past Performance Database (PPDB) • https://prod.nais.nasa.gov/cgi-bin/ppdb/home.cgi • Web Data Entry • Contract Number (NAS 5 XXXXX, S XXXXXY, NNG0XXXXXC) – Must be active contract • New eval or edit in-process eval • Verify contract data (changes to PSST) • Enter evaluation • Must have cage code or DUNS number • Enter CO signature date and save when evaluation complete (this triggers input into database) • Reports

15. Backup Samples

16. EXCELLENT Other: This attribute covers the contractor's performance in the area of the contract/business management function (including staffing and resources). NASA GSFC is very pleased with the level of commitment the contractor has in connection with the contract/business management function. The contractor is a Small Disadvantaged Business that exhibits a high level of commitment to meeting contract level requirements. As this is the contractor’s first contract with NASA, the many contract/business requirements (especially those associated with cost-reimbursement contracting) have the potential to overwhelm the small company, but the contractor displays an eagerness to learn how to do business with NASA and outstanding professionalism. It is a pleasure to work with the contractor’s business and financial team. In particular, NASA is pleased with the level of commitment that the contractor’s business management staff has exhibited in the area of reporting via the Contractor Cost Report (NF533 Monthly and Quarterly). NASA recognizes the contractor’s inexperience in this area and is pleased with the can do attitude that the business team exhibits. It is very important to NASA that it receive timely submission of the cost report data. This is essential as its contents are relied upon by senior management. The contractor has made significant improvements in this area and we are looking forward to their continued timely submissions. Source Selection Information—See FAR 3.104

17. VERY GOOD Timeliness: This attribute covers contractor performance in the area of compliance with delivery dates for contract level deliverables. In general NASA GSFC finds the contractor's performance to be fully responsive to contract requirements. One minor deficiency is noted, but it does not effect the overall performance. The Quarterly Status Reviews are slightly behind schedule, due to unforeseen and unavoidable events. Other contract level deliverables have been received in a timely manner. Source Selection Information—See FAR 3.104

18. GOOD Cost/Price: This attribute covers contractor performance in the area of price or cost control. In general, NASA GSFC finds the contractor's performance to be effective in this area. Source Selection Information—See FAR 3.104

19. SATISFACTORY Technical: This attribute covers contractor performance in the areas of compliance with contract requirements related to the spacecraft’s technical performance, as well as, quality assurance; and efficiency and effectiveness of the contractor’s program management function (including, management of staffing and resources). In general, NASA GSFC finds the contractor’s performance meets minimum acceptable standards. The Observatory has been integrated, and currently is in performance testing. Compliance with a majority of the contract’s technical requirements is yet to be verified, and the technical performance of the spacecraft is ultimately established after successful completion of the Observatory environmental test program. Several areas of concern are discussed below: … Source Selection Information—See FAR 3.104

20. POOR Timeliness: This attribute covers contractor performance in the areas of progress towards and compliance with contract delivery dates, as well as, the efficiency and effectiveness of the schedule management function. A poor/unsatisfactory rating is given as the contractor’s performance in these areas does not meet minimum acceptable standards. The contractor senior management failed to support its own program management team with adequate staffing, which led to significant delays in the C&DH (2 years), as well as most other activities across the program. Delay in completion of the C&DH resulted in a significant delay in completing Spacecraft Integration and Test (I&T), a major milestone event under the delivery order. The spacecraft bus and I&T teams were not prepared when the instruments arrived in the Summer of 2006. Spacecraft bus I&T started only 2 weeks prior to the arrival of the science instrument in July 2006. The original I&T plan had spacecraft bus I&T completion five months before delivery of the instruments. Many program activities are not well planned including Observatory comprehensive performance testing, fault management testing, Observatory power testing, and on-orbit scenario testing, ultimately leading to delays in completion of these activities. Launch site planning and preparation activities are at risk of falling behind. The common underlying issue is not the skills of the contractor engineers or management supporting the program, but inadequate staffing and resources in general. Source Selection Information—See FAR 3.104