1 / 41

US CMS Trigger Upgrade Proposal

US CMS Trigger Upgrade Proposal. Wesley Smith, U. Wisconsin Fermilab Directors’ Review 16 July 2013. L1 Trigger Upgrades : Detectors. Muon Detectors. Hadron Calorimeter. Electromagnetic Calorimeter. 401.04. Outline. Trigger TDR Status, Overall Goals

benita
Download Presentation

US CMS Trigger Upgrade Proposal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. US CMS Trigger Upgrade Proposal Wesley Smith, U. Wisconsin Fermilab Directors’ Review 16 July 2013 Fermilab Director's Review -- US CMS Trigger Upgrade

  2. L1 Trigger Upgrades: Detectors Muon Detectors Hadron Calorimeter Electromagnetic Calorimeter 401.04 Fermilab Director's Review -- US CMS Trigger Upgrade

  3. Outline • Trigger TDR Status, Overall Goals • Overview of Trigger Upgrade Design • Muon Trigger Upgrade • Calorimeter Trigger Upgrade • Overview of Cost & Schedule Fermilab Director's Review -- US CMS Trigger Upgrade

  4. L1 Upgrade TDR • Final presentation to LHCC was Tuesday June 11th • Public version of the TDR here:https://cds.cern.ch/record/1556311 • Given approval, although still awaiting final formal LHCC write-up Fermilab Director's Review -- US CMS Trigger Upgrade

  5. L1 Trigger Upgrade – overview • Trigger rates are driven by the increase in luminosity, the center-of-mass energy, & by higher Pile-Up • Detector readout systems limit Level-1 trigger output rate to 100 kHz, latency to 4 μsec. LHC beams collide every 25 ns. • Mitigate by improving e/γ isolation, τ id, μ pT resolution, μ isolation, jets with PU subtraction & L1 menu sophistication • Increase system flexibility with high bandwidth optical links and large FPGAs, and standardize on μTCA telecom standard • Build and commission upgrade in parallel with current trigger system to safeguard physics, decouple from LHC schedule • Overall design: • Two-layer calorimeter trigger with tower-level precision and PU subtraction • Integrated muon trigger combining all CSC, DT & RPC systems in track-finding Fermilab Director's Review -- US CMS Trigger Upgrade

  6. Final Overall Upgrade Scheme Fermilab Director's Review -- US CMS Trigger Upgrade

  7. Install fully paralleland higher bandwidthoptical path for CSC New CSC MPC mezzanine cards installed in cavern during LS1 Alleviates bottleneck and send all segments from each CSC (robustness to PU and collimated signals) Build up new Track Finder in 2015 and commission in parallel, ready by 2016 Endcap hardware design is proposed by CMS to be used for all muon regions: endcap, barrel, and overlap Muon Trigger Transition New system Old system Fermilab Director's Review -- US CMS Trigger Upgrade

  8. Muon Trigger WBS Detail → Calorimeter Trigger Fermilab Director's Review -- US CMS Trigger Upgrade

  9. 401.04.03 EMU Trigger Upgrade Cavern Counting Room Trigger MotherBoard Clock & Control Data MotherBoard Muon Sorter Clock&Control Sector Processors 401.04.03.04 Endcap Muon Track-Finder 401.04.03.02 Muon Port Card Mezzanine 401.04.03.03 MPC-EMUTF Optical Fibers μTCA: Advanced Mezzanine Cards from Telecommunications Computing Architecture (commercial telco, see backup slide) 401.04.03.05 EMUTF Infrastructure 401.04.03.06: Muon Sorter Fermilab Director's Review -- US CMS Trigger Upgrade

  10. 401.04.03.02 Muon Port Card Upgrade(Rice) (60 needed + 15 spares + 5 test setups = 80) x 60 +20 spares • 3 original optical links are still available • New mezzanine card with new FPGA and new links Fermilab Director's Review -- US CMS Trigger Upgrade Use the existing MPC main board Backplane interface to TMB remains unchanged

  11. Muon Track Finder processor: MTF7 13 needed + 4 spares + 3 test setups = 20 Optimized for maximum input from muon detectors (84 input links, 24 output links) Dual card with large capacity for RAM (~1GB) to be used for pT assignment in track finding Current prototype based on Virtex 6 FPGAis undergoing final tests (RLDRAM memory access) Virtex 7 FPGA design is ~75% done, expected late 2013 401.04.03.04: EMU Track-Finder Upgrade(U. Florida) Front: Optics card Back: Core FPGA cardwith PT LUT mezzanine Fermilab Director's Review -- US CMS Trigger Upgrade

  12. 401.04.03.05: EMUTF Infrastructure(U. Florida) 401.04.02.04 Endcap Muon Track-Finder 401.04.02.06: Muon Sorter Not US* *Processing the resulting tracks with RPC or CSC hits is a US responsibility Fermilab Director's Review -- US CMS Trigger Upgrade

  13. 401.04.03.06: Intermediate Sorter Upgrade(Rice) • Provide immediate CSC Trigger improvement in 2015 • Provide optical path to new μTCA Global Muon Trigger from old CSCTF from each endcap • Allows independent upgrade of individual endcaps • Add isolation to endcap muons from old CSCTF with a new mezzanine card on the old muon sorter to send an optical path to 2015 calorimeter trigger initial upgrade • Analogous to MPC upgrade: Fermilab Director's Review -- US CMS Trigger Upgrade

  14. Upgrade Cal. Trig. Layer 1 & 2 Hardware US: UK: Vienna: Fermilab Director's Review -- US CMS Trigger Upgrade

  15. Calorimeter Trigger WBS Detail ← Muon Trigger M&S paid byCMS-France Fermilab Director's Review -- US CMS Trigger Upgrade

  16. HCAL & ECAL Trigger Primitives • Upgraded HCAL Readout & Trigger Electronics (μHTR) is after split that sends data to old electronics (HTR) • μHTR supplies TP to upgrade cal. Trigger, HTR continues to send to present cal. Trigger • ECAL Trigger Concentrator Cards (TCC) are not upgraded, instead their mezzanines with trigger serial links are replaced (built by Lisbon): • Optical Serial Link Board oSLB replaces single copper link to current Regional Calorimeter Trigger (RCT – U. Wisc.) with two optical links (WBS 401.04.04.06) • One optical link to optical Receiver Module (oRM) on RCT (WBS 401.04.04.05) • Replaces copper Receiver Module • One optical link to input of Upgraded Calorimeter Trigger: CTP7 Card • 574 oSLB’s & oRM’s with Fibers installed in 2014 • French Contribution to Calorimeter Trigger (no US Upgrade Funds) • Presently oSLB & oRM in final prototype stage • oSLB: oRM: Fermilab Director's Review -- US CMS Trigger Upgrade

  17. ECAL Energy(TCC) HCAL Energy(HTR) Regional Calo Trigger Global Calo Trigger HF Energy(μHTR) HCAL Energy(μHTR) Current L1 Trigger System Upgrade L1 Trigger System Layer 1 Calo Trigger oSLB EM candidates Region energies oRM Layer 2 Calo Trigger Calo. Trigger Upgrade in Parallel: Split inputs from ECAL & HCAL Stage HCAL uHTR & assoc. Cal. Trigger Cards HCAL OpticalSplitters • Install optical SLB and optical RM mezzanines during LS1 • Install HCAL optical splitters during LS1 or YETS • Install HCAL backend μHTRcards for input to new trigger • Install HCAL frontend electronics after LS2 (finer long. segmentation) US US UK UK Fermilab Director's Review -- US CMS Trigger Upgrade

  18. 401.04.04.02: Calorimeter Trigger ProcessorVirtex-6 Prototype Board (CTP-6) – U. Wisconsin • 36 total + 8 spares + 2 test setups = 46 JTAG/USB Console Interface Mezzanine Power Modules MMC Circuitry Back End FPGA XC6VHX250T/ XC6VHX380T 4X Avago AFBR-820B Rx Module 12x Multi Gig Backplane Connections Front End FPGA XC6VHX250T/ XC6VHX380T Dual SDRAM for dedicated DAQ and TCP/IP buffering Avago AFBR-810B Tx Module Fermilab Director's Review -- US CMS Trigger Upgrade

  19. 401.04.04.02: Calorimeter Trigger Processor Final Version: Virtex-7 (CTP-7) – U. Wisconsin • Replace 2 Virtex-6s with a Virtex 7 for processing+ZYNQ for embedded TCP/IP endpoint • 30A, 1V power module for FPGA logic core • 3x CXP Pluggable modules for 36 Tx + 36 Rx 10G optical links • 2x AFBR-820 modules for 24 Rx 10G optical links • Simpler design to execute than the CTP-6 • 36 total + 8 spares + 2 test setups = 46 1V 30A Supply 3.3V Supply 1.5V Supply CXP Module 12Tx + 12 Rx 2.5V Supply CXP Module 12Tx + 12 Rx Virtex-7 VX690T FPGA 12X Rx ZYNQ XC7Z030 EPP 12X Rx CXP Module 12Tx + 12Rx (CTP-6 CAD View) Fermilab Director's Review -- US CMS Trigger Upgrade

  20. 401.04.04.03: Crosspoint IO Card: provides crate interconnections – U. Wisconsin (2/crate x 3 crates = 6 + spares) Controller (MMC and link mgmt) 4x4 Lane Bidirectional Multi Gig Backplane Connections 4X Avago AFBR-79EQDZ QSFP+ Module Positions Backplane Rx/TxRedriver ICs (top and bottom sides) Fermilab Director's Review -- US CMS Trigger Upgrade

  21. 401.04.04: CTP Infrastructure:Vadatech VT894 Crate Test Setup (Final system: 3 crates w/ 12 CTP7 ea. + 2 test setups + spare = 6) TTC Downlink U. Wisconsin designed backplane with dense card interconnects manufactured & installed in commercial Vadatech VT892 Crate available in Vadatech Catalog BU AMC13 UW CTP-6 UW CTP-6 UW Aux Vadatech MCH Fermilab Director's Review -- US CMS Trigger Upgrade

  22. Calorimeter Trigger Risk Mitigation • Goal (Original): • Provide readout of original RCT descoped during construction project • Present readout through GCT input buffer not workable with trigger evolution • Uses connection to a single CTP6 prototype or CTP7 card for DAQ readout • optical Receiver Summary Card (oRSC) paid by DOE Nuclear • VME Slave Interface Card • Fits in current RCT Crates (1 per crate) • 18 Cards in 2015 System • Receives RCT Jet Sum Card Outputto GCTon Copper “SCSI” Data Cables • Provides direct optical input to GCT • Bypassing old optical conversion cards • Planned use for Heavy Ion Triggers • Prototype under test  • Multiple optical outputs provide: • Inputs for upgrade calorimeter trigger &parallel operation of old & new trigger wherever ECAL & HCAL electronics notavailable in 2015 Fermilab Director's Review -- US CMS Trigger Upgrade

  23. ECAL energy Global Calo Trigger Regional Calo Trigger HCAL energy Evolution: Partial Upgrade in 2015 Uses reprogrammed RCT clusters with improved algorithms in Layer-2 Also, brings in all the power of finer grain HF using “Slice Test” of Layer-1 & 2 Improved PU subtraction, Isolation Calculation (e/γ/τ/μ) & Half-tower Position Resolution Add to bring in finer grain EM clusters US Items: oRSCs, oSLB/oRMcommissioning CTP Layer 1 Processors RCT RO+Testing CTP UK Items: MP7 Layer 2 Processors 36 mHTRs HF energy 2 CTP7s 9 CTP7s Layer 1 Processors oRSC oRM ECAL & HF Clusters Half-tower position oSLB RCT RO + Testing 2 CTP7s 4x4 E+H Clusters 2x1 E+H Clusters 2-bit region ID Spare Layer 2 Calo Trigger Layer 2 Calo Trigger Heavy Ion Muon 4 MP7s Jets & Sums eGammaTau GCT fallback and Minimal Stage-1 fallback remains available Layer 2 Processors With just a fraction of final cards or even prototypes, derive many benefitsof fullupgrade, incl. muon isolation Fermilab Director's Review -- US CMS Trigger Upgrade

  24. Trigger Staging • CTP7 M&S: no HB/HE Backend until 2016 • Delay purchase of 34 CTP7 cards until FY15 • Use 12 CTP7 Cards w/prototypes as additional spares & for test setups. • Connected as shown on previous slide • EMU CSCTF M&S: one endcap in 2015 & other in 2016 • Commission & test one endcap thoroughly so 2nd can be integrated quickly • Delay spares & use prototypes for spares, test setups • Compatible with Global Muon Trigger using different Endcap inputs • Stages and Content: Fermilab Director's Review -- US CMS Trigger Upgrade

  25. Technical Feasibility • Working prototypes for all major cards exist & have passed tests • Production quotes exist for all critical parts with confirmed delivery upon order • Upgrade is based upon common μTCA hardware platform also used by HCAL & other CMS systems • Common hardware components across subsystems: Virtex-7 FPGA, multi-gigabit optical links • Many handles to facilitate testing – test-pattern injection, spy-buffer readout as well as test stands • Upgrade trigger project replaces existing trigger hardware built by the same team → interfaces well understood • Same team wrote software & firmware for existing trigger system → scope & requirements well understood • Requirements are frozen in approved Trigger TDR • System is commissioned & operated in parallel with existing trigger Fermilab Director's Review -- US CMS Trigger Upgrade

  26. Trigger Project Organization Fermilab Director's Review -- US CMS Trigger Upgrade

  27. Trigger Management CMS Experience • W.S. (U. Wisconsin) • CMS Trigger Project Manager 1994-2007, • Trigger Coordinator 2007 – 2012 • US CMS L2 Trigger Project Manager 1998 – present • Darin Acosta (U. Florida) • CMS Trigger Project Manager 2012-13 • EMU Track-Finder, US CMS Trigger, 1998- present • Sridhara Dasu (U. Wisconsin) • US CMS L3 Manager for Calorimeter Trigger, 1998 – present • Author of original & upgrade cal. trig. Algorithms 1994 – present • Pam Klabbers (U. Wisconsin) • CMS Regional Calorimeter Trigger Operations Manager (more than a decade on RCT project) • CMS Deputy Trigger Technical Coordinator • Ivan Furic (U. Florida) • CMS CSC muon trigger maintenance & operations 2010-11 • CMS CSCTF Upgrade leader • Paul Padley (Rice U.) • US CMS EMU Project Manager 2006-2012 • EMU Port Card, Clock Card, Sorter, 1998 – present Fermilab Director's Review -- US CMS Trigger Upgrade

  28. CMS Trigger Engineering Experience • Tom Gorski (U. Wisconsin) – Calorimeter Trigger • Over a decade of engineering on the CMS Calorimeter Trigger • Alex Madorsky (U. Florida) – Muon Track-Finder • Over a decade of engineering on CMS Trigger, EMU, CSCTF • Mike Matveev (Rice U.) – Muon Port Cards & Sorter • Over a decade of engineering on CMS Trigger, EMU, Port Cards, Sorter • Mathias Blake (U. Wisconsin) – Cal. Trig. Firmware • New to project • 5 years Applications Engineer at Xilinx, 2 years FPGA Engineer at DRS Technologies, 3 years FPGA Design Engineer at Dolby Labs Fermilab Director's Review -- US CMS Trigger Upgrade

  29. C&S Methodology • M&S: • Parts based on quotes & prototype purchases • Ordering times based on experience with prototype orders • Board cost & delivery times based on prototype boards • Assembly cost & delivery times based on prototype boards • EDIA: • Labor Hours based on experience with corresponding tasks done for prototypes and on experience in delivering the original CMS trigger system (done by the same people & groups). • Labor costs based on actual labor rates of all of the individuals who are now working on the project • Contingency: • Follows US CMS Contingency rules Fermilab Director's Review -- US CMS Trigger Upgrade

  30. 401.04 Cost Estimate with Contingency Base Cost & Contingency in AY K$ Fermilab Director's Review -- US CMS Trigger Upgrade

  31. Schedule/Milestones • The project completes at the end of FY16 • FY14: • Launch production of trigger electronics 1QFY14 • Deliver on-detector muon trigger electronics to CERN: 3QFY14 • Deliver Stage 1 calorimeter layer 1 & muon track-finder electronics to CERN: 4QFY14 • FY15: • Test operation of trigger calorimeter and muon trigger electronics*: 2QFY15(*except muon sorter) • Deliver Stage 2 calorimeter layer 1 & muon track-finder electronics to CERN: 2QFY15 • Deliver Muon Sorter electronics to CERN: 3QFY15 • Test Operation of Muon Sorter electronics: 4QFY15 • FY16: • Test high luminosity capability of endcapmuon trigger electronics: 4QFY16 • Test high luminosity capability of calorimeter trigger electronics: 4QFY16 Fermilab Director's Review -- US CMS Trigger Upgrade

  32. 401.04: Trigger Resource Profile Base Cost = AY $K (No Contingency) Fermilab Director's Review -- US CMS Trigger Upgrade

  33. Overall Trigger Risks • Senior Engineer becomes unavailable • Hire new engineer, subcontract to consulting firm, use FNAL engineer • Funding is delayed • Commission with prototypes and/or fewer production boards • Software or Firmware does not meet requirements • Hire extra expert effort to recover schedule and help personnel • Boards are delayed (design, manufacture or testing) • Hire extra effort to speed up testing schedule • Vendor non-performance • Acquire spending authority to use alternative vendors (while original funds are being unencumbered). • Input or output electronics (non-trigger) delayed • Built in capabilities of trigger electronics provide signals for their own inputs & outputs • Overall Risk Mitigation Strategy: • Continue to provide fully operational current trigger system in parallel with upgrade commissioning • Partial operation of the upgrade systems provide tangible benefits from 2015 onwards Fermilab Director's Review -- US CMS Trigger Upgrade

  34. Trigger ESH&Q • Safety: follows procedures in CMS-doc-11587, FESHM • L2 Manager (W.S.) responsible for applying ISM to trigger upgrade. • Under direction of US CMS Project Management • Fermilab employees and CMS collaborators working at Fermilab & CERN receive training in accordance with Fermilab & CERN policies. Personnel conducting work at collaborating institutions are trained in accordance with the policies of those institutions. • Regular US CMS Upgrade Mgm’t meetings review work activity, schedule, costs, hazards, risks and controls. • Quality Assurance: follows procedures in CMS-doc-11584 • Regularly evaluate achievement relative to performance requirements & appropriately validate or update performance requirements & expectations to ensure quality. • QA: Equipment inspections & verifications; Software code inspections, verifications, & validations; Design reviews; Baseline change reviews; Work planning; and Self-assessments. • Graded Approach: • Apply appropriate level of analysis, controls, and documentation commensurate with the potential to have an environmental, safety, health, radiological, or quality impact. • Four ESH&Q Risk levels are defined & documented in CMS-doc-11584. Fermilab Director's Review -- US CMS Trigger Upgrade

  35. Cost & Schedule Understanding • C&S understood since prototypes for all major cards have been built & have passed tests • C&S based on experience of the same team that built original CMS calorimeter & muon trigger systems • Hardware is based upon common μTCA hardware platform also used by HCAL & other CMS systems • Same team wrote software & firmware for existing trigger system → scope & requirements well understood • Requirements are frozen in approved Trigger TDR • Schedule is robust because it always provides a working trigger at each stage with parallel operation of old & new trigger systems, commissioning & testing of new system during data-taking (with data) & evolution to final system while providing immediate availability of improved trigger in 2015 Fermilab Director's Review -- US CMS Trigger Upgrade

  36. Response to Recommendationsfrom May Review • Develop a plan prior to the CD-1 review for procurement of components that need to be installed prior to the end of LS1. • These components are the Muon Port Card Mezzanines and MPC-EMUTF Optical Fibers that must be installed in the collision hall during LS1. • These components are being moved to the operations program so they can be purchased in 2013. • Incorporate the missing information identified within comment section into the CDR prior to the CD-1 review. • This has been done and the CDR has been revised Fermilab Director's Review -- US CMS Trigger Upgrade

  37. Conclusions • Trigger Upgrade is technically feasible since based on working prototypes, & built by the same team that built original CMS Calorimeter & Muon triggers. • Trigger Upgrade can be accomplished within Cost & Schedule since based on working prototypes and experience in building the original CMS Calorimeter & Muon triggers. • Risks are understood and familiar due to team experience – risk is mitigated by parallel operation of existing and upgraded calorimeter systems • Plan to complete prototyping and begin production when approved. Fermilab Director's Review -- US CMS Trigger Upgrade

  38. BACKUP Fermilab Director's Review -- US CMS Trigger Upgrade

  39. Calorimeter Trigger Components Fermilab Director's Review -- US CMS Trigger Upgrade

  40. TMT Calorimeter Trigger Upgrade Time-multiplexed architecture, calo data from an entire event processed by a single processor Fermilab Director's Review -- US CMS Trigger Upgrade

  41. Pipelined Calorimeter Trigger Upgrade Each Layer 2 processor calculates suite of trigger paths To GT To GT Trig Path 1 (MP7) Trig Path 2 (MP7) Trig Path 3 (MP7) Trig Path 4 (MP7) Trig Path 5 (MP7) Trig Path 6 (MP7)  Crate G Crate G Up to 12 Trigger Path Modules Each Trigger Path MP7 rcvs 48 total 1st-Level fibers @ 9.6 Gbps (2/region) 2x288-fiber patch panel (2x24×12-fiber optical ribbons in & out) Layer 1 sends dedicated clusters to Layer 2 6 x 8-fiber ribbons for Lateral Network (@ 4.8 Gbps) Each 1st-Level CTP7 Drives 2 fibers out to 6-12 Trigger Paths @ 9.6 Gbps Calorimeter Trigger Processor (CTP) Card Crosspoint I/O (CIO) Card CIO-L CTP CTP CTP CTP CTP CTP CTP CTP CIO-U CIO-L CTP CTP CTP CTP CTP CTP CTP CTP CIO-U CIO-L CTP CTP CTP CTP CTP CTP CTP CTP CIO-U Crate A Crate B Crate C ECAL HCAL ECAL HCAL ECAL HCAL Fermilab Director's Review -- US CMS Trigger Upgrade

More Related