1 / 44

US CMS Silicon Tracker

US CMS Silicon Tracker. I&C and M&O planning Joe Incandela University of California Santa Barbara US CMS Silicon Tracker Project Manager July 22, 2005 Fermilab. Overview. There have many reviews: US CMS PMG Review of M&O: Fermilab, April 11-12, 2001

koren
Download Presentation

US CMS Silicon Tracker

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. US CMS Silicon Tracker I&C and M&O planning Joe Incandela University of California Santa Barbara US CMS Silicon Tracker Project Manager July 22, 2005 Fermilab

  2. Overview • There have many reviews: • US CMS PMG Review of M&O: Fermilab, April 11-12, 2001 • US M&O Evaluation Group Review: Fermilab, April 8-10, 2003 • US LHC M&O Evaluation Group Meeting: Brookhaven, February 5-7, 2004 • US LHC M&O Evaluation Group Meeting: Fermilab, January 27-29, 2005 • Will show excerpts from a few • Main message is that Sitrk planning has not changed substantially over the years • Will show overview of current M&O project file • Some I&C is included in the project file • Discuss current integration planning and a new initiative for remote effort

  3. Tracker Outer Barrel Silicon • Joe Incandela, University of California Santa Barbara • US CMS Silicon Tracker Project Manager • April 9, 2003 • Outline: • Basic considerations and assumptions • Project end and transition to M&O • Maintenance and Operation Plan

  4. CMS Tracker Outer Barrel (TOB) ~100 m2 Pixels End Caps (TEC 1&2) Inner Barrel & Disks (TIB & TID) 2,4 m 5.4 m volume 24.4 m3 running temperature – 10 0C

  5. Basis of estimates Experience considered: CDF Run 1b and Run 2a D0 Run 2a Scaling factor for resources ~ 1 despite large increase in channel and module counts Considerations TOB = CERN+US CMS but… CERN has large general infrastructure responsibility US CMS group is an obvious choice to play a lead role in M&O team. Operational expertise lies with those having the most testing experience during construction Very useful Tevatron experience Resource driving assumptions: US will assume a major portion of responsibility for TOB M&O Need continuous coverage of 6 main M&O areas Detector should be ready for physics as early as possible Basic Considerations (US CMS will now also produce and test >3000 TEC modules…)

  6. CDF/DO Experience CDF Silicon installation and commissioning group CDF Silicon Cables exiting the bore of the outer tracker “ Our current commissioning effort is taking so long partially because of inadequate resources (both physicist and non-physicist). So, I would advise that for commissioning especially, you do not underestimate the need for … resources - ask for enough dedicated engineers to complement as many physicists as you can get. ” -- co-leader of CDF silicon operations group in mid 2002. • CDF and D0 installation & operation • A very big job which takes many months and many people • Current (~steady state) operations requires on the order of 7 high level experts with backup of ~10 additional operators and monitors • We assume that some monitoring can be done remotely for TOB

  7. Project end & Transition to M&O • US must anticipate the need to cover much of the effort for the installation and commissioning (I&C) and probably the lion’s share of the maintenance and operation (M&O) of the TOB. • Some I&C resources thru FY05 (now FY06) are included in the construction project. • Installation/testing of rods in wheels • (Originally) scheduled for Jan 04 – April 05 • (Now scheduled for Sep. 05 – May 06) • 2-4 FTE Sr. Physicists, 7 Post-docs, students. • Testing modules on rods at CERN before installation • Limited system tests. • 1 FTE Engineer (base). • Rod & Module repairs, cooling system, mechanics. • Laboratory space and equipment for testing and repairs. • In late 2006, large M&O support team must be in place. • We estimate ~12 FTE physicists at CERN together with continued engineering and technical support & associated M&S and operating costs for maintaining a small laboratory space for testing and repairs.

  8. Ops. 2-3 phys. 1/2 eng. 1/2 tech. 24/7 on-call to avoid dead time. Leadership: organize US effort, executive decisions, interpret diagnostic results. Interface with CMS admin. DAQ 1.5 phys. 1/2 eng. 1/2 tech. Front end and higher level modules. FEDs and EB managers. Diagnostics, testing, (repairs, re-testing ?), swapping DAQ modules. Radiation protection and cooling ~1.5 phys. 1/4+1/4 engineer, 1 tech. Monitoring, repair. System issues: cooling tech present or on call at all times to deal with problems as they arise. 4. Crates and power supplies. 1.5 phys, 1/2 eng. 1/2 tech. Testing supplies prior to installation, retest after installation. Safety interlocks. A serious ongoing maintenance issue. 5. Monitoring and calibration databases 4 phys. 1/2 tech. at CERN Additional remote effort (see below). Need to catch problems in data and in calibration techniques. Very significant effort just to maintain a good channel list. 6. Alignment 1.5 phys. Laser alignment runs, hardware maintenance. Offline alignment and monitoring of alignment constants. M&O for Tracker Outer Barrel: 6 Main M&O areas

  9. Other areas • Not included in our resource projections: Module Failures analysis and development of failure prevention methods: • CDF/DZero have required considerable resources to • Understand why some silicon modules have failed • Example: Resonant wirebond breakage • Develop failure prevention strategies • For resonant wirebond breakage - developed completely new operating procedures to prevent repetitive readout sequences that led to exactly periodic current swings. • “If these resources are forced to come from operations, as was the case at CDF, things will be strained and commissioning and/or operational efficiency will be affected.” – CDF silicon operation group co-leader • Some of this can be done remotely (see discussion on FNAL remote monitoring/diagnostics station below)

  10. US CMS Silicon Tracker US LHC Detector Maintenance and Operations Evaluation Group Meeting Joe Incandela University of California Santa Barbara US CMS Silicon Tracker Project Manager January 28, 2005 Outline: Project overview and Status Schedule and Project Completion Installation and Commissioning, Maintenance and Operations R&D for upgrades

  11. Project end & Transition to M&O • We assume we will be needed to cover at least half of the effort of installation and commissioning (I&C) of the TOB. • Installation/testing of rods in wheels (05-06). • 2 FTE Sr. Physicists, 7 Post-docs & students. • Testing modules on rods before installation. • Limited system tests. • 1 FTE Engineer (base). • Rod & Module repairs, cooling system, mechanics. • Laboratory space and equipment for testing and repairs. • For maintenance and operation (M&O) of the TOB we need a team to provide support in 6 areas of operation. • By late FY06 early FY07, US M&O team must be in place. • We estimate 13 physicists at CERN together with continued engineering and technical support & associated M&S and operating costs for maintaining a small laboratory space for testing. Only substantial changes are in the schedule dates…

  12. Project I&C Project I&C now overlaps with M&O in FY06 but this is probably a good thing because more resources are going to be needed to recover schedule.

  13. M&O Cost Estimate

  14. US CMS SITRK M&O Resources • Manpower on Base • Post-docs, students, and one engineer.. • Post-docs all redirected from other programs or the production effort. • Manpower on Project • All technicians and additional engineering.

  15. M&O Thru 2009 (as of Jan. 2005) 1.1M$ COLA (using Jan. formula with 1.1 FCSH/USD) 1.2M$ Engineering + technical + ALL travel 350k$ R&D

  16. Experience matters First 2 years of operation (FY07-08) will be critical years • Difficult problems will have to be solved and will require the most experienced people • Establish all steady state operations of the tracker • Diagnosis and classification of many failure modes and problems • Detailed operating procedures & training guidelines • Refine all monitoring, calibration and alignment methods • Current uncertainties make it difficult to argue strongly that we absolutely need all of these resources. On the other hand, Tevatron experience tells us to prepare as strong a team as possible

  17. Upgrade R&D Issues • CMS silicon has limited lifetime. • 10 years of operation (we hope). • Further, possibly higher luminosity running will require more rad-hard silicon than currently exists. • R&D for current silicon took many years. • We need to be involved in future efforts at the level of prototyping and testing silicon or silicon-alternative designs. • 65k$ per year M&S • FY04 spent 90k$ for NRE and new sensor production • Demina (Rochester) Bortoletto (Purdue) • FY06+ • Some ideas now being developed. • Will visit HPK in October.

  18. Summary & Conclusions (Jan. 2005) • US Groups now responsible for building and testing roughly 65% of the CMS tracker modules • We have lost a year due to component problems. • Expanded capacity to recover schedule • Components now ok. • Based on CDF/D0 experience, significant resources needed for I&C and M&O. • We estimate 13 scientists plus engineering and technical support to allow US to have significant involvement in all aspects of operations Nota Bene: The US tracker group now has played a more important role in the construction project than originally anticipated or panned (e.g. problems found and solved with sensors, hybrids, rods… and a major increase in fraction of tracker modules built in US). Nevertheless, we have not made any significant changes to our M&O resource plans… We believe we have a good plan!

  19. TOB Project as of July 2005 • US TOB rod integration effort stalled • Serious problem found in US at both FNAL and UCsB • A very robust solution has been found (Option 7) • New interconnect cards are required (see next slides) • We pay 80k$ in order to expedite them (saves >1 month) • Total schedule hit is going to be around 5 months • Module production to continue • Need to purchase 400 sensors from HPK (250$) ASAP!!! • Need additional module storage costing ~15k$ • We are currently storing almost 2000 modules! • TOB integration to proceed • Use rods modified (Option 6) and fully tested in the US • September installation of one full cooling segment of TOB6- at CERN with substantial US involvement (we hope) • New Clean room (Tracker Integration Center – TIC) to be ready for support tube in October, fully operational by December (my estimates).

  20. Rods Overview • Basic components procured • Assembly of frames is DONE (modulo ICC) • Integration of modules on rods is understood • Rate of ≳ 8/d in the US is not be a problem … • Testing was problematic and rod production was halted • I2C communication errors seen on 30-50% of rods • Recent and very substantial effort to understand and remedy • Many thanks to : A. Marchioro, Wim Beaumont, M. Johnson, G. Maggazu, Slawek Tkaczyk… • Simple solution (number 6) an improvement but not quite.. • A better solution (number 7) adopted and requires new ICC • Secondary issues of header errors and failures in LT tests are also under study

  21. Main cause SDA Unequal fall-time of SCL and SDA signals SCL on FE-Hybrid Picture shows sequence of SCL/SDA signals with SDA pulled-down in AOH and measured on FEH

  22. Option 6 PSU FE FE-Hybrid PSU Control APV 82W SCL SDA AOH 82W Jumper creates a common time domain but some recent information from US indicates may still be marginal LLD

  23. Option 7 PSU FE FE-Hybrid ICC PSU Control 100pF DCU/APV CRT245 (Line driver) 100 pF 330W SCL AOH 22W SDA Parasitic on AOH CA 22W LLD 10pF

  24. Effect of “damping” resistor SDA with “damping” SDA without “damping” SCL not show on this slide

  25. Current (limited) production • While I2C solution 7 is being prepared we are assembling and testing a limited number of rods with "jumpers" (I2C Solution 6) • Test I2C solution 6 • To be shipped to CERN to populate 1st cooling segment, and possibly the cosmic rack • Gain experience with (somewhat) large scale rod production • Somewhat longer testing cycle to double-triple-quadruple-check I2C behavior • Gain experience with testing protocol • Document/investigate non-I2C failures • Refine cut requirements

  26. Part Count as of July 13, 2005

  27. 5 with unconnected thermistors (as read out by DCU, i.e., these are not the "hardwired" thermistors) 4 at FNAL, 1 at UCSB 1 fails ring redundancy test FNAL 1 Laser failure FNAL 1 with strange problem, open ring? UCSB, under investigation All 4 SS4 rods (UCSB) have unusually high noise SRT results – out of 43 tested rods Discussed in TOB meeting at July Tracker Week: agreed upon actions to be taken

  28. But also… • Still low rate of I2C errors 230 that we thought would have been cured by jumper (Solution 6) • Other not-yet-understood I2C errors • A few other annoyances like header errors HW or SW problems? Experts are actively investigating

  29. After this "limited production" we will stop Take stock of situation, firm up procedures Attack other (non I2C) problems Hold a mini-workshop to get the UCSB and FNAL-based people together in August We will likely consider another fairly substantial production exercise Allows one to hunt more rare problems Module storage… Rod Plans

  30. Meanwhile …Wheel is ready! Wheel on assembly chariot Thermal screen operational Wheel ready for insertion in the tracker support tube and… The tube with thermal screen are already in the assembly clean room.

  31. Preparations for TOB integration - where we are as of July 14, 2005 He leak test stand OKRod precabling stand OKCu power cables two weeksCable “bridge” one weekVacuum system for soldering to be commissionedService supports in productionCooling manifolds in productionReadout hardware FEDs/FEC available - firmware 1 week (?)Power supplies availableTest software to be commissioned on C-Rack Plumbing in 186 (cooling and He) Basically done DB tools defined - only ribbon registration implemented ------ later ------Alu cables need to update layout and start production (lengths OK)Cable test box in production in Lyon (mid-August) - need updated pinoutDOHMs finalizing tail lengths - start production

  32. US involvement • New key personnel • Jeff Spalding – will become deputy manager responsible for overseeing initial US integration activities at CERN • Very experienced manager • Assist the talented but a bit inexperienced CERN team • Improve US-CERN communications • Slawek Tkaczyk- will initially lead the US testing effort during integration at CERN. • Also very experienced • Will oversee experienced US physicists at CERN during integration • September ~1 month integration Exercise • A strong US team at CERN under Spalding and Tkaczyk • Two experienced US rod testing physicists • Two experienced US Silicon technicians/engineers

  33. Goals of September Exercise • Establish detailed procedures • Establish rod acceptance tests (upon receipt from US) and rod tests prior to installation on wheel • Full installation and commissioning of a rod on wheel • Serial installation/commissioning of up to ~20 rods • Removal sequence • Unsolder all installed rods and remove • Batch installation • Methods/procedures for batch mode installation of rods • Slot in place up to ~10 rods and test connectivity (front ends not powered) • Solder cooling pipes all rods and test cooling ring • Cable and test all rods and test DAQ ring(s) • Use this information to develop resource loaded schedule for full TOB integration • We expect to be revising schedule more than once, as other problems develop in commissioning. Information from all integrated experience will be used together with resources/time available to optimize safe assembly…

  34. Integration in 2006 • After the September exercises… Plan to have US team start to appear in late ‘05 or early ’06 • Depends upon restart of rod production – If smooth, we can transfer some of the experts to CERN. • Current management planning is directed toward evolution from sub-system integration teams to full tracker integration team. • We are already working with the integration manager (Nicola Bacchetta) to develop TOB integration planning documents that will be reviewed in Autumn. • Idea is to develop a uniform set of protocols followed by all subsystems. • By the time we have completed the subsystems’ integration we would like the transition to commissioning of the full tracker to be as smooth as possible. • This is also facilitated by the new tracker integration center.

  35. Current Status of TOB • Module production lines have completed 25% of TOB • >99% yield and >99.95% good channels • Production of 40-50 modules/day can be achieved, provided that we have adequate parts flow and stable testing • Rod frames production in Helsinki is complete • Rod integration and testing capacity ≳8 rods/day • I2C communication problem is understood • Rod integration and testing has restarted in the US with solution 6 • Solution 7 likely definitive – will learn more in coming weeks. • TOB Wheel is ready and other preparations are in train • Integration exercises will start soon • Very capable team at CERN • US may send a team in September (headed by Jeff Spalding)

  36. Tracker DAQ Test Facility at FNAL Being organized by M.Johnson, J. Spalding and S. Tkaczyk. (S. Tkaczyk provided these slides – modified somewhat by J. I.) • Motivation: Allows US CMS tracker community to: • Participate in online code development for monitoring data quality and detector performance • Experience is crucial for the tracker to be usable early in running! • Educate new people in using the CMS DAQ HW and SW tools • Much more effective to learn this before taking shifts at CERN! • Study long term performance of strips and pixels • Could determine our upgrade strategy • Troubleshoot tracker problems emerging during regular operation of the CMS tracker • Pathological modules/hybrids found and fully categorized in US production efforts will be used as a “library of pathology” that could be useful for development of work-rounds for problems arising during data-taking.

  37. TrackerDAQ@FNAL: Configuration • HW - a replica of the CMS Tracker DAQ slice to read out a set of RODs (and pixel structures?): • Components: FEDs, FEC, TTC, PS, cables, fibers, DCS, interlocks. • SW – Run Control, XDAQ shell with tracker-specific extensions for data acquisition, DCS, PS control, calibrations, online data analysis

  38. TrackerDAQ@FNAL: Remote Data Quality Monitoring • Allow CMS collaborators to participate in the operation of the tracker in the US. • Develop expert level knowledge among US CMS tracker community by means of: • Operation of the elements of the tracker (strips, pixels) at FNAL CMS DAQ test facility • Development of online software in collaboration with other tracker teams • Participation in daily operation and monitoring tasks at CERN through regular shifts at FNAL

  39. TrackerDAQ@FNAL: Implementation • Dedicated consoles and workstations (costs included in the Remote Operation Center) • Scheduling/oversight of regular tracker shifts at FNAL – 1/4 FTE • DAQ test stand at SiDet • Will find some eqpt. at FNAL and get some on loan from CMS. (To purchase all is > $100k). • HW support – ¼ FTE • PC, Software Support – ¼ FTE Comp. Prof. • Total Physicists: 6 at FNAL and 3 at CERN • Would include post-docs/students from university groups • Other items: • Dedicated audio/video equipment at CERN and FNAL • Travel Support

  40. Preliminary (crude) estimates • FY06 • $20-50K -DAQ Hardware • $10K -Audio and Video Equipment • $25K -Analysis Workstations and Data Storage • $25K-50K - travel support – (5-10 people) • 1/2 FTE support – technician + comp. professional • FY07-FY09 • $50K/year – travel support • $10K-$15K/year - maintenance

  41. Conclusions • TOB module production is on schedule • Rod production halted to fix major problem • Integration planning is underway • Schedule must be revisited (this will not be the last time…) • US beginning to structure a strong team at CERN • Jeff Spalding the US tracker project deputy in residence • Slawek Tkaczyk will oversee US testing team • Critical integration exercise at CERN in September • US I&C and M&O planning is stable and adequate • Key will be to see if we actually get the support we need to fulfill our plan which has been stable for many years. • TrackerDAQ@FNAL: Early stages of planning for an FNAL-based training/monitoring/online-development facility for the CMS tracker • An excellent and necessary US tool!

  42. Additional HPK Silicon for TOB

  43. Qty for various scenarios

  44. Proposal • Cost per sensor is 428 Euros • 171,200 euro @$1.22/Euro • 207k$ • 20% for currency fluctuations & contingency etc…  250k$

More Related