Powering tests before LHC warm-up Second LSC meeting

1. Mirko Pojer and Matteo Solfaroliwith inputs from A. Verweij, S. Le Naour, J. Steckert, V. Montabonneton behalf of MP3, EPC, QPS, PIC and software colleagues Powering tests before LHC warm-up Second LSC meeting

2. Outline • Scope of the tests • The strategy • The preliminary results* • PC performance • Conclusions * For a more detailed analysis, refer to the TE-TM of 19/03/13 or LMC next week (A. Verweij)

3. Scope of the tests Fewweeks wereallocated for “special” powering tests, to detect possible limitations other than the splices for the design operation of the superconducting circuits (after LS1 there should be no known constrain to the 7 TeV target)

4. The planning

5. The powering ‘bible’ • Preparatory meetings took place already at the end of 2012 • A document was prepared, which describes the tests to be performed, the parameters to be used (in agreement with ABP), the acceptance criteria, as proposed by MP3 • Preliminary preparatory work: • LSA parameters change • Change of HW limits for PC • Creation of a new test campaign (including MTF) • Minor modification to the test execution GUI • Verification of testing chain (acc GUI, PM analysis, MTF)

6. The strategy • Tests performed via the powering sequencer • Support of experts: • EPC and PIC on call • Continuous presence of MP3 at the CCC --> decisions taken on the spot • Daily meetings and ad-hoc MP3 discussion to steer the tests and threat relevant issues • EPC and QPS piquet for tunnel interventions

7. What did we test? • 540 circuits were tested, including IPQs, IPDs, ITs, 600 A and some 120 A circuits, (almost) all powered up to 7 TeV equivalent current • 773 successful tests were executed in 10 days • More than 1300 tests performed (including repetitions due to quenches or other kind of faults) First week-end Second week-end No test!

8. In detail • Individual tests were performed on: • ITs and IPQ/Ds • ramp up to 7 TeV equivalent current (asymmetric for IPQs), 30 min at FT, FPA at the end (without heater firing) • 600 A circuits • current cycle up to 7 TeV equivalent current, 30 min at FT (positive and negative) • ramp up to 7 TeV equivalent current and FPA at the end • Some ‘weak’ 120 A circuits • current cycle up to 7 TeV equivalent current • As a final validation of each sector, the powering of all circuits of a sector (PGC) was performed, with the main dipoles and quadrupoles ramped at 4 TeV and the rest at 7 TeV. After 2 hours at flat top (mainly to study the impact on current leads cooling) a fast power abort was initiated, to investigate coupling between circuits • (first time of ‘most’ of the circuits powered together to 7 TeV)

9. The preliminary results

10. Inner Triplets • All ITs were commissioned to 7 TeV, with a very limited number of quenches (no quench up to 6.75 TeV) • IT.R1 – weak insulation of one QH • A discharge of the heaters was performed at 4 TeV • After analysis of the discharge, MP3 approved the powering to 7 TeV • Powering was successful --> No further action is needed during LS1 • IT.L5 – warm cables are too resistive • The ramp rate of the two high current modules was reduced • The circuit was successfully commissioned --> Replacement of the water cooled DC cables (2000 mm2for RQX and 1000 mm2 for RTQX2) during LS1 (ECR - EDMS#1215705)

11. Individually Powered Quadrupoles • The QPS detection threshold of all RQ9, RQ10 and the RQ8 in the injection regions had been modified in the winter stop 2011/12 (400 mV instead of 100 mV), to enhance EMC immunity --> massive re-conditioning and re-qualification were needed before testing the circuits, thus delaying their commissioning --> QPS will re-cable all RQ8, RQ9 and RQ10 to reduce EMC disturbance, in LS1 • All IPQs were successfully tested up to 7 TeV equivalent current • Many quenches were observed, above all on the MQM @4.5 K (the 14 magnets experienced, in fact, 49 quenches) • RQ5.R1 had two unprotected quenches • After a first quench, the PM from QPS was missing • Analysis of the PC post-mortem was pointing to an undetected quench • Quench detector was found in a frozen state, without communication (since Aug ’12, after a thunderstorm) • ElQA validation was carried out to assess the electrical integrity of the magnet • After an additional quench, the circuit reached 7 TeV equivalent current • An incident report will be written --> Perform already planned replacement of all DQQDI detectors by nDQQDI detector boards during LS1 (based on different firmware/algorithm which excludes the failure mode observed; new detector stays latched after trigger, thus enforcing a reset to continue powering)

12. Individually Powered Dipoles • All IPDs reached 7TeV equivalent current, except RD3.L4 which reached 6.9 TeV; its commissioning was stopped after several quenches to avoid degradation of the magnet, but a clear monotonous training is observable, which made MP3 confident that it could reach 7 TeV • RD2.R8 successfully commissioned to 7 TeV • In 2008 strongly limited to 6.8 TeV (quenched three times at the same current level) • At the beginning of this test campaign, the same limitation appeared • After further analysis by MP3 colleagues, it turned out that the quench was in the bus-bar (in the external link joining the DFBMB with the magnet) • Radiographies showed that the bus-bar might have suffered from bad cooling • The link height was modified and the magnet successfully powered up to 7 TeV --> During LS1, it would be maybe wise re-checking all links

13. 600 A circuits • ‘Most’ of the 600 A circuits were successfully tested up to 7 TeV equivalent current • 407 circuits tested, with some 80 quenches experienced (some of them at FT – analysis on-going by MP3) • 3 circuits saw their nominal current reduced by a small amount (to be further analysed by MP3 and current level to be agreed by ABP) • All ROD/Fs and RSD/Fs (in 4 over 8 sectors) were commissioned to 590 A (vs. 550A), upon request of OP, to help against beam instabilities at 7 TeV or special optics needs --> Further increase is submitted to the change of the QPS DCCTs (most probably not relevant for LS1) • RSD/Fs (in 4 over 8 sectors) were tested at a higher ramp rate for special optics needs • All 600 A leads of ITs needed overcooling: resistive leads without flow-meters --> reinforcement of the action already launched for consolidation during LS1 (flow-meters with feed-back control) • All 600 A circuits of ITs were commissioned to 550 A, but are presently limited to 350A due to nested powering --> Pending analysis and decision with ABP colleagues whether 350 A are sufficient for 7 TeV operation; on the opposite case, action to be launched for a software powering interlock to be implemented during LS1 • Special extraction tests performed for MPE studies

14. 600 A circuits • RCBXH3.L5B1 was successfully qualified to nominal current: • This circuit had been first powered in 2008, but since 2009 was never powered due to an incomprehensible fault • After some tests and analysis by experts, ElQA swapped two voltage taps and the circuit could be ramped up to nominal --> No further action needed in LS1 • Successful investigation on RCO.A78B2 • The circuit was never used due to a suspected splice defect • Powering with ‘binary search’ by ElQA in the tunnel, allowed to localise a high resistance spot between magnet C19L8 and B20L8 (3 ICs, 1 magnet, 2 connections busbar/magnet) --> SMACC-SIT will have to open three ICs to possibly identify a weak splice --> In case of undetectable fault, the magnet could be by-passed by means of a spare busbar

15. 80-120 A circuits • 34 x 120 A circuits were individually powered up to 7 TeV (all 120 A circuits were powered during PGC) • All but one showed a ‘reasonable’ behaviour (further analysis by MP3 needed) • RCBHYS5.R8B1 could guarantee a reliable operation only at a ‘low’ current – to be further analysed by MP3 and assessed by ABP --> in case of need, the possibility of replacement with a warm magnet (as done in 8L) could be investigated • All 120 A circuits of ITs were successfully powered up to 7 TeV (with a minor limitation on one of them), but the leads needed to be overcooled --> Consolidation in LS1, as for the 600 A circuits

16. PC performance

17. PC failures • A series of unexpected faults was recorded on the power converters • Trips due to 0V Zero Crossing (2x), traced back in a bad contact of internal power converter output current measurement (LEM) used only for the 0V crossing --> possible consolidation to be performed on all modules • Trips due to IOUT Over Current (Output Current > I_HARDWARE) after new setting of the circuit hardware current limits , probably due to oxidation contacts (not used since 2009) --> oxidation problem to be addressed in LS1 • Trip of 42 sub-converter failures (total population = 741); most of the times with a trip of the entire converter Sub-converter failures are related to output module rectifier diode failures (diode dying in short) Issue already presented by Y. Thurel at TE-TM57 See more details on V. Montabonnet’s at TE-TM -19/03/13 --> Action to be taken by EPC to replace diodes

18. Diode failures • Finish failure data analysis • Continue electrical recheck on actual design • During LS1 1. Change all the output module rectifier diodes 2. Add fans for a better cooling of the output modules to improve lifetime and MTBF 12000 diodes 1695 output modules • Big investment (personnel and budget) for LHC tunnel as well as for test bench

19. Concluding remarks • Very good preparation of the tests • Software colleagues • EPC • MP3 was essential (together with the inputs from BE/ABP) • Some (usual) steering issues at the beginning • Accesses (VIPs….) • QPS firmware update • Few surprises • Many quenches (where is the memory of these circuits) • RQ5.R1 issue • PC failures • Many positive aspects • Most of issues are identified and actions are on the way • No (?) critical cases in the machine (maybe one corrector…) • Overall result is incredibly good and promising for after LS1 • Once more, an impressive common effort by all teams involved!