Rods schedule and issues
1 / 27

Rods Schedule and Issues - PowerPoint PPT Presentation

  • Uploaded on

Rods Schedule and Issues. J. Incandela University of California Santa Barbara for the TOB Rod Group Tracker Integration Meeting October 27, 2005 Slides courtesy of D. Abbaneo, C. Campagnari, A. Dierlamm, Y. Gotra, J. Lamb, others…. Rods Overview. Basic components procured

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Rods Schedule and Issues' - ewan

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Rods schedule and issues
Rods Schedule and Issues

J. Incandela

University of California Santa Barbara

for the TOB Rod Group

Tracker Integration Meeting October 27, 2005

Slides courtesy of D. Abbaneo,C. Campagnari, A. Dierlamm, Y. Gotra, J. Lamb, others…

Rods overview
Rods Overview

  • Basic components procured

  • Assembly of frames is DONE (modulo ICC)

  • Integration of modules on rods is understood

    • Rate of ~ 8/d in the US is not a problem …

  • Testing was problematic and rod production was halted

    • I2C communication errors seen on 30-50% of rods

    • Very substantial effort to understand and remedy

    • Solution required making new Interconnect cards (ICC).

      • Now in production.

        • First 25 sets arrive in US today (?)

        • Larger batches arrive at CERN starting early November (?)

      • Rod production scheduled to restart this week.

Recent rod production
Recent rod production

  • Meanwhile we implemented a simple fix and built ~75 rods

    • Simple fix allows long term testing in Multi-Rod Test (MRT) stands

    • It is not robust enough to be a final solution

  • We shipped rods to CERN for integration studies

    • For integration studies

      • Develop/document procedures for installation/removal of rods


  • Assembly procedures well developed

  • Trained personnel (including backups) at both sites

  • Takes ~ 1-1.5 hours to assemble a rod

    • Add 0.5 hour for ICC swap

  • Not a bottleneck

    • Long term testing ~6 rods/2 days or 4/day if necessary

      No issues – everything is in place

Electrical testing
Electrical testing

  • General idea

    • Assemble rod

    • Quick test of single rod

      • Single Rod Test stand (SRT)

    • Long term test

      • long is really not that long (~ 1-2 days)

      • Multi Rod Test stand (MRT)

Single rod test srt
Single Rod Test (SRT)


  • Tests of

    • Hardwired thermistors and humidity sensors

    • CCU ring redundancy

    • DCU readouts

    • Leakage current

    • Pedestals & noise

      • Just looking for gross problems

      • No channel level characterization

    • At room temperature


Srt continued
SRT (continued)

  • Most tests done using LtStruct

    • currently V30

  • Hardwired sensors tested with simple custom box

  • Root macro similar to module test macro generates plots

  • Plots etc. uploaded to e-log

  • Output root and text files saved for ALL tests

    • including version number, if tested more than once

  • Only piece missing: protocol for testing digital functionality, experts to provide (?)

Multi rod test stand mrt

(Semi) long term test

~ 1-2 days

6-8 SS rods/load

Thermal cycle

Full channel level characterization

Multi Rod Test-stand (MRT)



Original mrt test cycle have since developed shorter one
Original MRT test cycle -have since developed shorter one













Mrt status
MRT status

  • Hardware: all OK, robust

    • Several different coolant leaks at UCSB in last few weeks – hopefully now solved

  • DAQ software (LtStruct)

    • Much progress in the last months

    • Thanks to great cooperation from Wim et al.

Mrt status cont
MRT status (cont.)

  • Analysis SW

    • DefectAnalyzer

      • Makes plots and XML file

      • First pass cuts defined

        • Will iterate using data from early production

    • Custom perl script

      • Compares bad channels on rod with bad channel lists from module tests

        • Also flags DCU problems, etc

    • Custom root macro

      • To make plots similar to defect analyzer

      • OK, but will be retired

Workshop in august
Workshop in August

  • We had an internal workshop in August

    • Try to converge on remaining loose ends

    • Minutes

  • Some of the issues discussed

    • Original default long term standard scenario

      • 2 day, go cold 4 times

    • "Finalize" cuts to flag bad channels

    • Module grading - same as ARCS

      A: < 1% bad channels

      B: < 2% bad channels

      C: > 2% bad channels

      F: bad HV

Workshop continued
Workshop (continued)

  • Rod grading:

    • Pass: only A or B modules

      • ≥ 1 out of 3 non-DCU thermistors OK

      • I(leak) for all groups < 20 A

    • Fail: otherwise

  • XML file for DB

    • Went over it line-by-line

    • Decided on several changes additions

      • e.g., added thermistor information

      • For details see minutes of workshop

    • Now implemented

      • Still a few problems uploading

Towards production what was missing
Towards production: what was missing

  • Demonstration that we can sustain production rates on a daily basis for a long time

    • To this end, we were thinking of going through a production exercise, starting asap

      • ~ 40 "type 6" rods/site

  • ICC Swap – where will this be done?

  • Definition of digital tests

    • May have already been completed ?

  • Some DB details

    • Should be OK

Rod status now ucsb and fnal
Rod Status Now - UCSB and FNAL

  • While waiting for ICC’s to start production, the two sites have focused on slightly different issues

    • FNAL: testing of new SW (FEC V3, Scientific Linux, Lt V30)

    • UCSB: production exercise to shake down remaining operational issues

  • Fine-tuned scenario to match 4 rods/day rate

    • Freezer takes max 8 SS4, 6 SS6, 4 DS  need one day scenario (so much for long term  )

    • Only two thermal cycles WCWCW

    • CalProfRuns only with inverter-on to save time

    • Added HV cycling at each temperature

      • This is not working reliably, may have to ditch


  • 4 sets (2/site) of OFECs in transit from CERN

    • FNAL will "commission" this system first; UCSB will follow when it works 

  • First batch of ICC cards for 25 rods also in transit

    • Held up at Memphis, expected delivery - today

    • ½ FNAL, ½ UCSB

    • Will mount on existing rods, then put modules on rods, then test, then ship. ASAP

Fnal status 1
FNAL Status (1)

  • Over 30 production scenarios run on MRT since July 2005

  • 50 SS6 rods and four DS4 rods were assembled

    • (using jumpers on pre-production ICC cards), tested and burned-in

    • Migration of rod test software running to Scientific Linux 3.0.5 with FEC software V3 was successful

    • Long term testing with latest release of the FEC Software showed good stability of software and the control ring

      • 1 upload/download error in 24 hours with 4 DS4 rods

    • Latest release of the LtStruct (v30) was extensively tested

      • Number of production scenarios with Single and Double Sided rods

Ucsb production exercise mechanical
UCSB Production Exercise: Mechanical

  • Built 24 rods last week (22 SS6, 2 DS)

  • Warmup 1st two days, then built 4 rods/day

  • Bottom line: OK

  • Some Issues:

    • Dirty rod boxes, Al dust

    • Tail bending a la FNAL: OK.

    • ICC cards swap: adds 20-30 minutes/rod

    • DS rods: worry about exposed non-encapsulated bonds

      • Designing cover with cutouts for ICC cards to enable handling of connectors and cables with minimal risk

Ucsb production exercise testing
UCSB Production Exercise: Testing

  • 20/24 rods through SRT

  • Mounted some modules with known pinholes

    • Flagged them without a problem 

  • 1st MRT scenario OK, 2nd failed

    • Failure in newly introduced HV cycle.

    • CAEN got stuck. Not a new problem.

    • A couple of laser failures (?) – Worrisome…

      • Currently under investigation – expect info later this week.

    • Some noise oddities to be investigated

    • Some ICC/header errors.

      • Hopefully will be better with new SW, new OFEC

    • (Almost) successful 1st upload of rod XML file to DB

      • Need some small name changes

Ucsb production exercise short term plans
UCSB Production Exercise: Short Term Plans

  • Stop making rods

    • We learned what we needed to learn

  • Continue exercising testing cycle

    • HV cycle

    • XML uploads

Rod reception status
Rod Reception Status

  • Preparing rod reception testing for high rate

    • Automating flagging of bad channels and validation comparison to US

      • Installed/running DefectAnalyzer with the same cuts as US

      • Bad channels agree so far

      • Mostly concerned with making sure there is no shipping damage or other exceptional problems

  • All but two of the forty “production” rods from the US have been tested

Rod reception status1
Rod Reception Status

  • High current encountered on some modules

    • Annoying but not fatal

    • Few modules on rods from US exhibit high current - humidity effect with a longer “memory” than usual

      • A module that has been exposed to room air for several hours will show a current ramp-up to up to 1mA, a few minutes after bias turned on

      • Then the current falls back to normal levels over a time of about an hour

      • If the module is kept in dry air the high current doesn't reappear if the bias is turned on and off

      • Seems to be correlated with a problem in mixing the Sylgard 186 encapsulant, which left a “stain” on the sensors.

Rod reception cern typical noise
Rod Reception CERN Typical Noise


  • Lots of progress

    • Can build and test rods in the US

  • Some problems

    • Production startup will be a little rocky

    • Hope to converge to stability by end of year, prior to peak production rates

    • Robustness of stands has not yet been fully tested, and we will likely encounter some problems and short delays over time.

  • Schedule

    • On paper, we can get all rods here in time

    • In practice this looks extremely challenging.

      • I would expect that a more realistic schedule would have 1-2 months contingency