Some initial thoughts on issues relating to Large Facilities

1. Some initial thoughts on issues relating to Large Facilities Mark Coles DRAFT – To review with Tom Cooley

2. High and low level issues • Strategic issues – maybe use a future revision of the facilities guide as a lever to address these policy issues: • Funding for initial planning and technology development • Parallel development of a user community • Narrow definition of scope • Life cycle cost accounting • Tactical issues • Building relationships with the IG and the Hill • Project management planning and execution with projects now underway or just starting • Facilities guide outreach and enhancement to add additional topics • Procurement strategy and assistance

3. How do large projects receive funding prior to approval? • Some large projects require significant resources early in their development to support • Enabling technical development • accurate bottoms-up cost and contingency estimates • WBS dictionary and resource loaded schedule • Development of mature procurement strategy and specification • Pre-award funding for technical development can be substantial (tens of millions of $), and should not tax the base program. Examples of programs that have benefited from pre-construction awards for significant technical development are: • LIGO (support for 40 meter interferometer on Caltech campus), • Ice Cube (AMANDA, and further back DUMAND), • Ice Cube received specific Congressional funding for phase 1, which was not requested by the NSF • ALMA ($32 million for competitive engineering design of antennae).

4. MRE Strategy:Development of a user community • How does the user community get developed and funded? • LIGO may have been unique since there was no previous broad experimental community. Many experimentalists came from other fields, and some had not previously had NSF support. • Need to have early and sustained participation of broader community. Where does funding come from to support this, since otherwise there are potentially program breaks if community has to wait until operation to propose? • MRE relationship to base budget, • facilities often enable small researchers to do research at the facility

5. “The colors of money” • Narrowly defined scopes for individual budgets can hamper a project by not permitting flexibility to respond to new technical developments or other opportunities. • Examples: • grid computing at LHC • Participation of community in installation and commissioning of a facility • EPSCOR funding (NEES) • A central feature of successfully reviewed MRE projects is the potential for outreach and broader societal impacts, yet this is not an allowed expense for MRE budget. Is there an opportunity to address these issues?

6. Life Cycle Costs • How should full life cycle costs be presented? • Successful projects will have significant cost growth as user community utilizes and adds scope to initial project. • This should be represented as success. • Cost accounting needs to distinguish this from cost and schedule overruns with respect to initial scope. • Development of formal, visible process for project scope, schedule and budget modification within NSF – maybe this already exists? • Full cost accounting should tell an accurate story of the project. A stock prospectus provides accounting information from which an investor can infer the status of the company and where it is headed. Life cycle costs should allow one to accurately identify successful projects and distinguish them from projects that are merely expensive.

7. Cost reporting • Need a transparent and systematic process within NSF to track changes to scope, baseline budgets, reportable milestones, etc. • How should upgrades be accounted for? • Should operations labor be included in upgrade cost accounting? • Defined process for reporting, addressing, and mitigating cost escalation when a project gets outside the budget envelope. – add to guide? • Commissioning and initial operation should be included as part of MRE since they are often part of the validation of the facility. Since real boundaries may not be clear, imposing accounting boundaries is artificial. Can this be changed?

8. Future Costs • Can a “liens list” be created and managed within the NSF to anticipate and support full life cycle costs for each project? • This would be a current list of possible future expenditures, such as maintenance,contingency repairs and spares, upgrades, possible scope expansions, etc. • Done at project level in LIGO, maybe helpful to know at NSF level

9. Tactical Issues • Facilities Guide revision after some initial period to include policy issues: • Seed money to develop project before it goes forward – “phase 1 funding”. • Funds for development of user community. When an MRE project enables a community, a lot of small research grants will utilize the facility and increase proposal demand on the base program • Greater guidance on mitigation options when projects do go off track • Coaching within NSF and outreach to research community will be needed to help projects develop reasonable WBS and resource loaded schedules and procurement strategies, and to use them to manage what they do.

10. Development of cooperative agreements • Work with awardee to develop • Scope • WBS requirements • Staffing plan • Integrated resource loaded schedule and major milestones • Reporting strategies to NSF • What is reported • Reporting periods • Review schedule for major procurements, design, demonstration, and construction,

11. Project Management Strategies • Should minimize risk – include in Facilities guide some “how-to” and follow up with on-site coaching on additional topics: • Structure projects so that there is a tiering of authority with a central management. A single awardee holds financial and managerial authority over subawardees, and holds project contingency at the highest level • Examples: • LIGO has Caltech as awardee and MIT as subaward – worked well • NEES – 18 awards(!), one critical award to system integrator has encountered some problems • Earthscope – 4 awards, plus USGS (unbudgeted) • Not simple when foreign or external agency participation • Provide coaching to project where appropriate

12. Procurement Strategy • Work with awardee to develop procurement strategy, schedule, cost and schedule contingency estimates. • Tiering of subcontracts to maximize FFP use with well defined deliverables. • Minimize subcontractor couplings (GFE, schedule linkage) to avoid interference. • Limit in-house work to areas where scope and uncertainty are within capabilities of performing organization • Major procurements require timely evaluation and approval at NSF • LIGO received excellent assistance from Dave Berley plus Aaron Asrael, Jesse Laskin, Karen Sandberg, Carol Langguth. Who are the present equivalents? • Assemble an “A Team” of equivalent personnel to provide rapid project response? (Maybe already exists, but need to get to know these staff and process for working together.)

13. Contracting • Specific provisions for contracting in LIGO went well beyond standard provisions required in OMB. These were negotiated by Aaron Asrael and Ed Jasnow and cooperative agreement was amended appropriately • Monthly cost/schedule reporting, formal reviews, CDRLs, etc.

14. Other items addressed in LIGO that might be discussed in contracting part of Facilities Guide • Establishment of cash-neutral payment milestones in contract schedule of values • Minimize Level of Effort contracts and time and materials contracts • Construct a resource loaded schedule that allows one to extract estimates of cost impact for material and labor rate variances, and schedule slip, =d$/dX • How to construct set of project milestones that is sensible – tells an accurate story of the development plan and minimizes false alarms. • Develop a project QA strategy that includes witnessing key subcontract activities, material acceptance testing, • Subcontractor development, submittal, and acceptance of a subcontractor QA plan • Transition plans for going from construction/commissioning to operation.

15. More guide topics • Define when and how external Technical Review Boards should be created by awardee to develop and update plans, specifications, award contracts, etc. • Same for Contract Review Boards • Define when a field manager is required to oversee various activities – such as construction management • Define relations between contractor and next lower tier • Materials receiving and qualification • Travelers and other in-process documentation • Policy on drawing and specification release, control, and revision

16. WBS Development • Guide should address: • Methodology for development of WBS • Methodology for development of contingency. • A clear definition of what is meant by contingency: • work detail that is more effective to address through field changes or in process changes than it is to design in advance. • Average uncertainties in cost estimating • Guidelines for estimating contingency • Define how contingency should be held within a project – at the top level – and not distributed over the WBS elements.

17. Guide Enhancements • Coach project management to set up: • CCB process within project that is transparent to all interested parties • Monthly (when appropriate) cost/schedule reporting within project and disseminated within NSF • Document Control Center that seamlessly links project documentation and NSF documentation electronically • Use Cooperative Agreement as framework to introduce these commitments within each MRE program

18. Additional ideas for additions to the Facilities Guide • How a project gets started and how the NSF wants to work with the community in the early phase of development • Elaboration on the Internal Management Plan - How a project is managed inside the NSF – scope additions, EHR involvement, interdivisional interactions, what the program officer does, PO oversight within the NSF, etc. • NSF expectations for how project should be managed during construction • Guidelines for development of a cooperative agreement – reporting, procurement guidelines and approvals, definition and visibility of milestones, etc. • How is overall procurement strategy developed and reviewed? • QA, safety, elaboration on property management • How to handle overruns and deficiencies • Guidance structuring reviews during various phases of project life – composition of review board, financial, technical, management reviews

19. More ideas… • Preparation for operation • Within NSF – advance budget planning, impact on user community • By performing org • Operational readiness review (include safety) • Acceptance test • Pre-ops • Thoughtful choice of milestones to identify key activities • Operations • What NSF expects of the performing org to begin ops • Budget and staffing exercises • There is no operational contingency, yet major repairs are a fact of life and appropriate “management reserve” needs to be held somewhere, even though this is a hot button issue with IG • What happens within NSF to enable a community through operation of a facility • Termination process – lots to cover here – just one page in guide

20. Relationships • Build a constructive relationship with Cheh Kim and others on the Hill • Gary Sanders impression that there is an opportunity to build on the very positive view Kim has of Tom and BFA • A “heads up” where appropriate, to let Cheh know what activities are going on with respect to MRE would be very helpful, according to Gary • Be open and transparent to OIG • Develop visible management information system of tracking cost, scope, and schedule to allow rapid and easily obtained assessments of project status, funds expended, etc. • Proactive interactions?

21. Existing issues • NEES is in trouble – NCSA is under-performing and scope/schedule/deliverables are not defined. If I were PO I’d ask for an audit before continuing. • Earthscope project management looks troublesome since structure replicates deficiencies of NEES • Michigan State Superconducting Cyclotron ops budget is way out of line (+40%), and on-site review provided little information to substantiate why they need more money. • What should be my involvement in these issues?

22. New programs I need to become familiar with: • Not yet met with: • Underground Lab • Oceans • Antarctic • Observe Lehman review at DOE • Get to know Jim Yeck - Observe DOE LHC monthly review at FNAL • ALMA visit prior to construction start

23. Budget planning • How does budget process work for this position? • FTE’s • Travel money • Supplies • Office space, etc. • Do I need to develop a plan for next FY?