Closeout Report for the BigBOSS Review Lawrence Berkeley National Lab December 6-7, 2011

1. Closeout Report for the BigBOSS Review Lawrence Berkeley National Lab December 6-7, 2011

2. Want to thank the lab and the collaboration for hosting and preparing for the review. Closeout sections: Scientific Technical (4) Cost, Schedule, Management Each section will have: Findings Comments Recommendations

3. Review Charge • The purpose of this review is to assess the scientific and technical goals and feasibility • of BigBOSS; and to assess the technical aspects, cost, schedule and management for • the R&D planned during the current pre-conceptual design phase. • 1. Scientific: Are the scientific goals compelling and attainable? Are they • justified in relation to other existing and planned experiments? • 2. Technical: Is the current pre-conceptual design and associated implementation approach attainable and likely to satisfy the scientific goals? Have the major risks been identified and is the R&D plan in place to address them? • 3. Cost and Schedule: Are the estimated R&D costs and schedule reasonable and adequately justified? • 4. Management: Is the appropriate management structure and team in place for the R&D effort in the pre-conceptual design phase?  

4. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.1 Findings • 1.1.1 The collaboration’s proposal is to conduct a spectroscopic redshift survey aimed at measuring the Baryon Acoustic Oscillation signal over a unique redshift range, an important component in the portfolio of dark energy experiments. • 1.1.2 The radial BAO signal will measure H(z) over the redshift range 0.8<z<1.6 and fill a hole in planned measurements of the expansion of the Universe through BAO exactly at the epoch in the Universe when the dark energy component is becoming dominant. • 1.1.3 In addition to the direct measurement via BAO, the collaboration will use the redshift survey to measure redshift space distortions, a measure of the growth of clustering in the Universe.

5. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.1 Findings • 1.1.4 The collaboration will also measure the BAO signal at lower and higher redshifts, providing better constraints on cosmology at these redshifts than “currently operating” stage III experiments such as BOSS, Wigglez, and HETDEX. • 1.1.5 The collaboration has made a compelling case that the successful completion of their survey would represent the state-of-the art in BAO measurements, and would represent at least a factor of 10 improvement in the BAO contribution to the DETF Figure of merit compared to the current results from SDSS-III. • 1.1.6 The DETF “figure of Merit” resulting from the proposed project meeting its redshift measurement goals is competitive with the BAO components of space-based stage IV proposals such as WFIRST and EUCLID.

6. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.1 Findings • 1.1.7 The proposed survey aims to measure the galaxy power spectrum to 1% precision, which will constrain other fundamental cosmological measurements, from the combined neutrino masses to the validity of general relativity. • 1.1.8 The success of the proposal’s science goals depends on the successful identification and selection of target galaxies, and then on the successful measurement of redshifts of these targets. • 1.1.9 The collaboration has identified PTF as the primary source for the most critical target list, the ELG sample. They have established an MOU with the PTF project and are actively participating as a partner institution in PTF-2.

7. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.1 Findings • 1.1.10 The proposal has identified clear avenues to selecting targets for the ELG sample, the QSO sample, and about 75% of the LRG sample. The team is actively testing the validity these avenues using data from PTF, SDSS and WISE. • 1.1.11 The collaboration is acquiring data and performing simulations to verify that the proposed BigBOSS capability will successfully measure redshifts for the targeted sources. This R&D work is also extremely important for validating the science goals of the Survey. • 1.1.12 The collaboration has made a strong case that the science is attainable given small performance gains from designs of currently running projects (BOSS, LAMOST, SDSS-III).

8. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.1 Findings • 1.1.13 Progress is being made in understanding how to compare power spectrum measurements with cosmological predictions at non-linear scales (a process necessary to interpreting the redshift space distortions and broad band power spectrum measurements), but theoretical work will continue to be necessary before these measurements can be exploited. • 1.1.14 The Ly-alpha forest BAO technique remains an unproven method but the first published results from BOSS that measured the broadband flux transmission power spectrum have not revealed any systematic show-stoppers to date. • 1.1.15 The BigBOSS collaboration and the DES project are investigating additional dark energy gains from common analysis over the ~5,000 square degree overlap between the two surveys.

9. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) 1.Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? 1.2 Comments 1.2.1 While demonstrating the BigBOSS FOM from BAO is comparable to the BAO FOM from other planned projects is helpful in explaining the importance of BigBOSS, any such discussion should be clear about exactly what is being compared, particularly if the other project involves other techniques that are not accounted for in the comparison. 1.2.2 A more dark-energy-model independent test of the validity of GR would measure the curvature fluctuation generated per unit dynamical mass.  This requires a comparison of the redshift space distortions from the survey with measurements of space curvature,typically from gravitational lensing, and is most powerful if done in the same field. Perhaps this is where a DES/BigBOSS overlap can be very valuable.

10. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.2 Comments • 1.2.3 The progress made on validating the proposal’s plan for target identification is extremely impressive. In the year since the non-advocate review of the proposal by NOAO, the team has succeeded in addressing almost all the concerns raised by that review in terms of target selection, significantly reducing the risk to the project. • 1.2.4 It is important that the R&D work proposed to verify the target selection continue. In particular, the FORS project for ELG verification. It is important to test out the alternate options for acquiring the imaging data necessary for the part of the proposed BigBOSS project that does not overlap the SDSS footprint.

11. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? 1.2 Comments 1.2.5 The limitations and biases of Lyman-alpha forest measurements of BAO are not completely characterized, but the BOSS data will provide a good test-bed for these studies. 1.2.6. The collaboration should continue its participation in PTF2, because that dataset provides the target selection for the most critical portion of the survey.

12. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.3 Recommendations • 1.3.1 Continue developing tools to model nonlinear contributions to redshift space distortions and galaxy biasing prescriptions to achieve the required precision. 1.3.2 Continue to address Ly-a forest astrophysical systematics for the BAO measurement using the BOSS data as it is made available. 1.3.3 Study the impact of other systematics in addition to star-galaxy correlation on the large scale galaxy power spectrum measurements. 

13. Science & Survey plan Hu, Dell’Antonio, Albrecht (SC1) • 1. Scientific: Are the scientific goals compelling and attainable? Are they justified in relation to other existing and planned experiments? • 1.3 Recommendations • 1.3.4 Address the extent to which mock catalogs and simulations will be necessary for survey optimization and error analysis and the required computational resources. • 1.3.5 Complete the analysis of the FORS emission line galaxies to verify the selection criteria for ELGs and provide a test of the sensitivity of BigBOSS • 1.3.6 Use both simulations and data from BOSS to investigate the redshift-measuring efficiency expected for the ELGs.

14. Optics and Mechanical Olivier, Dell’Antonio (SC2) Findings (1 of 2) • A reference design for BigBOSS has been developed that encompasses: • — Corrector Optics/ADC/Barrel • — Hexapod/Cage/Thermal shroud • — Spiders/Upper ring • — Interfaces • CAD and FEM models were developed, and utilized for a • design study of the reference design in the following areas: • — Mass • — Volume • — Deflections and birefringence • — Dynamics • — Thermal environments • — Integration and alignment • The R&D plan proposes to increase fidelity of the models and continue to use them to analyze these areas to reduce risk.

15. Optics and Mechanical Olivier, Dell’Antonio (SC2) Findings (2 of 2) • Optics manufacturing plan, lens cell design, alignment strategy and associated tolerances were derived based on experience with DECam. • Further R&D work is proposed to verify optics manufacturability with vendors and to develop prototype subscale lens cell assembly. • The hexapod rotation range is +/- 2 degrees but it wasn’t known if this range is available during operations, that is, is the required focus, tip, tilt available with this rotation range and is this enough to compensate for telescope misalignment.

16. Optics and Mechanical Olivier, Dell’Antonio (SC2) Comments (1 of 2) • The reference design for the optical and mechanical elements and associated implementation approach appears feasible. • The major risks have been identified and an R&D plan is in place to address these risks. • The team has the necessary technical expertise and management structure to accomplish the proposed R&D plan. • Cost and schedule were presented/reviewed separately, but team correctly identified thermal analysis as major R&D area. • The optical design goes beyond the current state of the art, so additional design studies to optimize reference design in order to potentially save cost and/or time are warranted and are planned in the proposed R&D work.

17. Optics and Mechanical Olivier, Dell’Antonio (SC2) Comments (2 of 2) • Without a large angle field rotator, it’s probably worth computing if the available hexapod range is sufficient to adjust for telescope polar alignment around the sky, axis problems, etc. This adjustment could be done by repositioning the fibers, but rotation is probably easier.

18. Optics and Mechanical Olivier, Dell’Antonio (SC2) Recommendations (1 of 1) • Hold project meeting to review lessons learned from DECam experience regarding optics and mechanical issues following commissioning of DECam. • (e.g., was there any problem caused by not bringing the optics and the focal plane together earlier in the project, which might suggest a different strategy for BigBOSS) • A rotation requirement should be developed with KPNO or, conversely, a scheme for using a rotation offset in fiber positioning operations, perhaps using a DEC, HA table.

19. Focal plane system (including fibers and actuators) Uomoto, Olivier (SC3) 2. Technical: Is the current pre-conceptual design and associated implementation approach attainable and likely to satisfy the scientific goals? Have the major risks been identified and is the R&D plan in place to address them? The telescope focal plane is high-risk and will require significant R&D. The team is aware of this and has done a good job in identifying the risks by exploring fiber actuator options and planning a sub-sized focal plate build and analysis. The team is sensitive to throughput issues and it was good to see self-induced pressure to reduce the number of fiber connections. There were some omissions (e.g., how to deal with dead fibers) that indicate a tighter consultation with science is needed to generate requirements for things like the dead fiber count. The pre-conceptual focal plane design is close to conceptual design in most areas and the presented implementation will likely meet scientific requirements.

20. Focal plane system (including fibers and actuators) Uomoto, Olivier (SC3) • Finding: The team is aware of and sensitive to the installation problems associated with the 5000+ fiber cabling. The system design did not take into account the ease of replacing or repairing broken fibers. An analysis on how many dead fibers were allowed before a repair intervention was missing. • Comments: While fiber failures should be rare, accidents happen and it’s essential to build in repair plans from the beginning. • Recommendations: (1) determine from the science requirements the allowable number / configuration of dead fibers, (2) develop the scheme for replacing fibers at the focal plane, (3) develop the plan for replacing spectrograph fibers.

21. Focal plane system (including fibers and actuators) Uomoto, Olivier (SC3) • Finding: Fiber positions in the focal plane are tuned up using a focal plane image of the object and fiducial fibers. The plan for injecting light into the slithead fibers was not explained well and it was not shown that the required photon rate at the focal plane was deliverable. • Comments: This is a simple scheme that should work well if the spectrograph fibers can deliver enough light to the focal plane. • Recommendations: Apply R&D effort (1) for a concept to illuminate the slithead fibers and (2) determine that this concept delivers enough light to the fibers for the required camera exposure times.

22. Focal plane system (including fibers and actuators) Uomoto, Olivier (SC3) • Finding: The fiber viewing camera is more pre-conceptual than the other subsystems so there are a larger number of unexplored areas. • Comments: This equipment is part of the serial time budget for field setup so retiring this performance risk early, or handling the actual performance consequences, is important. • Recommendations: Research the proposed camera choice (Canon 400 mm) to show it performs well, understand the mechanical stability of the mounting location at the top of the primary baffle (is it stable against wind shake?), etc.

23. Spectrograph and Sensors and control O’Connor, Uomoto (SC4) • Findings • A thorough study of spectroscope design alternatives was shown and a baseline design has been chosen. • BigBOSS will require ten three-channel, 500-fiber spectrographs. • The spectrograph design has many common features with the MUSE system and LAM is working closely with the manufacturer of that system. • The Schmidt committee concerns about thin dichroics have been addressed. • Detector requirements can be satisfied by existing CCDs: the BOSS CCD is baselined for the visible and red channels, and a commercial CCD has been identified for the blue channel. • ASIC-based frontend electronics are proposed for video signal processing and two options are being pursued for clocking and biasing. • Four areas of CCD R&D are proposed: low noise readout, UV-enhanced window process, thick substrates for NIR sensitivity improvement, and buttable packaging. • A well-developed plan for instrument control and DAQ, based on BOSS and DES experience, was presented.

24. Spectrograph and Sensors and control O’Connor, Uomoto (SC4) Comments • A pre-conceptual design for the LED fiber illuminator was not presented. • The LPT cooler for the spectrographs is an unconventional choice and has potential cost, reliability, and vibration risk. • The baseline detector choices seem sound and are low risk. • A dynamic range requirement is not yet defined for the CCDs and readout. • R&D on CCD packaging does not seem to address a high-risk aspect of the project. • The instrument control team relies on algorithms for fiber view camera remapping and actuator repositioning to be provided. Recommendations • Perform a trade study on spectrograph cooling. Consider in particular the cryo/vacuum system developed for MUSE by ESO. • Define the dynamic range requirement for the CCDs and readout electronics.

25. Computing Lyon, Goodman (SC5) • A) COMPUTING AND SOFTWARE INFRASTRUCTURE • FINDINGS: • The actual computing requirements have yet to be determined. • The needed computing infrastructure (computing hardware, databases, system layout, etc) is not particularly challenging and is an evolution of the BOSS infrastructure. • The total data store size is small and is limited by the number of fibers and survey size, which are stable. CPU needs seem to be modest. • Useful simulations can be performed on your laptop. • It seems that in the past the software and computing effort were loosely organized and that was good enough. • COMMENTS: • Better computing and software management is advantageous. • There are at least five years before you start taking data, and computing technology and techniques will most likely change during that time. • Following software development processes and promptly writing documentation and process descriptions are crucial for a smooth running software effort.

26. Computing Lyon, Goodman (SC5) • RECOMMENDATIONS: • Determine the computing and software requirements mentioned in WBS 2.B.1 (Data system R&D) according to the R&D plan, starting with the physics goals of the experiment. • Create an organized computing and software development effort, with clear lines of project management and responsibilities. The effort should include the computing and software infrastructure, algorithm development, and simulations. • Investigate roles that computing and software professionals can play in the data system and pipeline development effort. Perhaps utilize expertise from other fields that are present at the labs/universities. • Continue the use of a centralized point for the software repository, documentation, and issue tracking. If TRAC is not performing well, consider evaluating and migrating to a new system in a prompt manner. • Keep abreast of and evaluate new computing techniques and be ready to adopt high impact improvements (e.g. can adapting the pipeline to GPUs give you a big advantage?). 26

27. Computing Lyon, Goodman (SC5) • B) SPECTRA EXTRACTION ALGORITHM • FINDING: • The spectra extraction algorithm used for BOSS is not sufficient for BigBOSS because, • the new experiment will examine fainter objects, • will have stronger susceptibility to the sky background, • and will attempt the Lyman alpha forest analysis that requires a more accurate spectra model. • A new more complicated “2D PSF [Point Spread Function]” extraction is under development and is a promising solution to these issues. • COMMENTS: • The NOAO review singled out the large computational needs of the new algorithm. Those estimates have since been shown to be largely overblown and the concern no longer exists. • But the central importance of the new algorithm to achieving the experiment’s results cannot be overstated. • It appears that a very capable team is attacking this problem.

28. Computing Lyon, Goodman (SC5) • RECOMMENDATIONS: • WBS 2.B.2 (Pipeline Reductions R&D) should be executed promptly. Make use of realistic simulations and spectra from other surveys (e.g. BOSS) to evaluate the algorithm. If a show-stopper exists, that must be determined quickly. As in the previous recommendation set, keep abreast of new computing technologies that could have a big impact on speed, maintainability, etc. • Work with NOAO to ensure that the official pipeline does not preclude community science 28

29. Computing Lyon, Goodman (SC5) • C) SIMULATIONS • FINDINGS: • There exists a mature end-to-end simulation of BigBOSS as well as other simulations that have already played an important role in the design and development of the experiment. • COMMENTS: • These simulations seem to be well developed and in wide use by the experiment. Continue this effort. • No specific recommendations

30. Cost & Schedule Green, Goodman (SC6) 3. Cost and Schedule: Are the estimated R&D costs and schedule reasonable and adequately justified? Yes, the estimated costs for the next 2 years of R&D are reasonable and will serve to mitigate some of the larger risks to the experiment.

31. Cost & Schedule - Findings Green, Goodman (SC6) FINDINGS • Strong support was expressed for BigBOSS by NOAO management • There was strong support expressed for this survey instrument at a Community Science Meeting. • Strong support was expressed for BigBOSS by LBNL management • Good support from LBNL for the management team and the project has been shown. • R&D key activities were identified. The estimated cost was $2.7M for FY12 and $3.5M for FY13, increasing as CD-0 was assumed at the start of FY13 and CD-1 in the third quarter of FY13. • A workable baseline design was presented which serves as a reference for cost and schedule. • MOA for R&D have been established for the major foreign partners of BigBOSS. 31

32. Cost & Schedule - Findings Green, Goodman (SC6) FINDINGS • The BigBOSS estimated cost is derived from the current WBS for R&D. L2 managers were identified. • The organization chart for R&D was shown and followed the WBS L2 structure. • The DOE base cost for the Project is estimated at $26.4M, without contingency. • Foreign costs are estimated to be $23.9M in the DOE metric and include the corrector, focal plane and spectrograph which are complete systems. • A technically limited schedule was shown in broad brush and the Critical Path (CP) was identified. Commissioning occurs early in FY18. The corrector is the CP. 32

33. Cost & Schedule - Comments Green, Goodman (SC6) COMMENTS • A cost estimate for R&D in FY12 and FY13 was shown. The level is about 10% of the Project cost which is roughly the expected scale. • A schedule with Milestones at L1, L2 and L3 should be developed, even for the R&D phase. The cost and schedule should be derived from the resource loaded WBS. • The Interface Control Documents (ICD) are very useful in tying the subsystems of the Project together. However, there is a time dependence and ICD milestones would be a useful tool to employ. • Change control should be thought through prior to CD-0. • Operations costs should be estimated prior to CD-0. • Installation and Commissioning tasks for BigBOSS should be defined and responsibilities assigned between BigBOSS and NOAO. Viable options for different operations funding modes appear to be possible. 33

34. Cost & Schedule - Comments Green, Goodman (SC6) COMMENTS • Given the prior performance and experience of the BigBOSS team, a ~ 35% level of contingency seems appropriate – for the DOE costs. • Operations costs/funding should begin prior to the end of the Project. • The committee believes that the R&D program will lead to a mature technical design within 18 months. • The maturity of the collaboration is an opportunity to move rapidly toward CD-0 and CD-1. 34

35. Cost & Schedule - Recommendations Green, Goodman (SC6) RECOMMENDATIONS • Develop the resource loaded cost and schedule and the other documentation to a point sufficient to request CD-0 approval as rapidly as possible. 35

36. Management Green, Goodman (SC6) 4. Management: Is the appropriate management structure and team in place for the R&D effort in the pre-conceptual design phase. The BigBOSS experiment is at the pre-conceptual R&D phase, which is prior to CD-0 or mission need. At that level the management team and the management tools which are in place are entirely adequate and are, in fact, in advance of what is normally required.

37. Management - Findings Green, Goodman (SC6) FINDINGS • The Project Management team which has been put in place for BigBOSS is highly experienced and has begun to set up the appropriate tools. An R&D and a preliminary Project organization chart were shown. • The funding for BigBOSS will have a large foreign component. They represent an opportunity since the foreign partners supply a very substantial fraction of the funding. • There are several Institutions in the US and outside the US that need to coordinate for BigBOSS to succeed. • A high level schedule was shown, with CD-0 at the start of FY13. • A WBS was shown with ~ 1000 entries • Major risks were identified. 37

38. Management - Comments Green, Goodman (SC6) COMMENTS • The R&D proposed for the next 2 years will mitigate major risks to the BigBOSS experiment. • Value engineering should be pursued beginning in the R&D phase. Readiness reviews should be scheduled and documented prior to major procurements and follow up action items should be tracked. • The managerial structure and MOU connecting NOAO and BigBOSS should be understood in detail prior to CD-0. • MOUs with the foreign BigBOSS collaborating institutions should be begun since about ½ the funding for BigBOSS is tied to foreign contributions. • DOE should be proactive in helping BigBOSS secure international commitments. 38

39. Management - Comments Green, Goodman (SC6) COMMENTS • The ICD, the System Engineer, timely tracking and reporting and the application of contingency will help manage the foreign contributions. • Risks due to foreign contributions are large and sufficient contingency should be assigned. On the other hand, the funds contributed by the collaborating institutions and the foreign collaborators could possibly allow for forward funding and thus “earn” schedule. • The “vertical integration” or factorization with the major foreign contributions responsible for L2 subsystems should make the management task easier since interdependencies are minimized. • Tracking and reporting tools, especially with foreign contributions, should be in place by CD-1. • Scope contingency and the associated science impact should be identified, both up and down. When the cost experience of the Project becomes known contingency can be applied to minimize the scientific impact. 39

40. Management - Recommendations Green, Goodman (SC6) RECOMMENDATIONS • Ensure that the science to technical requirements flowdown is conducted and publicized within the collaboration early in the R&D phase to enable an optimized experimental design • Ensure that the baseline design at CD-0 has been optimized within the science requirements and that the current reference design is not the last iteration. • Explore incremental costs and schedule and science impact in using alternative telescopes for CD-0. • Work with the BigBOSS stakeholders and the NOAO, NSF and DOE to bring BigBOSS management to a condition where CD-0 approval can be requested expeditiously and CD-1 could then follow quickly. 40

41. 41

42. Review Panel Members Andy Albrecht ajalbrecht@ucdavis.edu Ian Dell’Antonio ian@het.brown.edu Jordan Goodman goodman@umdgrb.umd.edu Dan Green dgreen@fnal.gov Wayne Hu whu@background.uchicago.edu Adam Lyon lyon@fnal.gov Paul O’Connor poc@bnl.gov Scot Olivier olivier1@llnl.gov Alan Uomoto au@obs.carnegiescience.edu Agency Attendees Kathleen Turner (Chair) kathy.turner@science.doe Jeff Pier jpier@nsf.gov Jim Ulvestad julvesta@nsf.gov Jim Whitmore jwhitmor@nsf.gov