QRL compensation bellows and main LHC cold boxes repairs – post LS1 summary

1. QRL compensation bellows and main LHC cold boxes repairs – post LS1 summary CRG-TM – 01 June 2015 Krzysztof Brodzinski

2. Outline • QRL repairs • Introduction – first failure, QRL standard cell layout • Failure mechanism • Repairs summary – lesson learned and conclusions • 4.5 K cold boxes repairs • Description of failures and repairs (case by case) • Repairs summary – lesson learned and conclusions CRG-TM_K.Brodzinski_2015.06.01

3. Introduction – first sign of weakness CRG-TM_K.Brodzinski_2015.06.01

4. Introduction – standard cryogenic cell Cell 15R4 (where the first problem appeared during Xmas stop 2011/2012) ~ 107 m QRIOA Internal tubes connection type: QRIOA QRIOA QRICA QRIAA Service module Pipe element Fixed point element Internal fixed point Internal fixed point (or vacuum barrier) Internal tubes connection type: end-end end-end end-end end-end compensators sleeves end-end end-end compensators sleeves Thermal contraction of ~54 m  162 mm weld Magnet cell 14R4 Magnet cell 15R4 There is 567 interconnections equipped with compensation bellows on all QRL lines. CRG-TM_K.Brodzinski_2015.06.01

5. Failure mechanism crack He (P, T) Metallographic observations (EN-MME) A “virtual” leak (not seen during global leak test) Global leak tightness still guaranteed by a ply Filling of the inter-ply space with time (3-4 years of operation) Pressure increase of the inter-ply space during warm-up  compensator collapsing /leak creation ! Maxi pressure with conservative assumption: isochoric transformation and space initially at line conditions (P, T)  up to ~1000 bar for Line C Contribution from L. Tavian QRL vacuum signal CRG-TM_K.Brodzinski_2015.06.01

6. Failures geography • When the failure mode was understood -> trigger for x-rays of all line C compensators • During review of line C x-rays it was seen that line D is affected as well (but not other QRL lines) -> trigger to review all line D bellows, some additional x-rays were necessary • Trigger for spares order for line C, E and D 14R4_line C CRG-TM_K.Brodzinski_2015.06.01

7. Repairs – preparation and comments • Preparation of necessary documentation and drawings – available on CRG server and EDMSWent fine, all necessary drawings were available at CERN, no necessity to contact AL • Build up of a mock up in 276Went very slow on the beginning, no enough coordination of the activity. Accelerated and well guided from July 2013 after increase of the bellows number to be repaired • Spares (5 line C, 7 line D, 5 line E) tightness test by VSC – developed application of special boombing test for test of interplies volume tightness (only 3 line C bellows were OK) – exact data available via VSCCollaboration with VSC in full understanding and with friendly atmosphere. • X-rays campaign, identification of all faulty connection and number of spares needed Collaboration with EN-MME and Qualitec went fine, with good timing to anticipate the repairs activity. Need of 16 spares confirmed -> Too less number of spares at CERN to cover all failures, necessity of installation DN80 on line C in some places, at least 2 spares leaky. • Visit to Witzenmann on 6 August 2013 – C. Garion (VSC) and K. Brodzinski (CRG) – discussion on spares order and design improvement Discussion very useful to react quickly for additional spares, difficult negotiation for the design improvement, difficult commercial discussions for the price and delivery time, late deliveries wrto initial agreement. Finally bellows delivered by Witzenmann with improved designed (modified weld design and 1 ply tight and all remaining vented to the vacuum). Two other contacted firms did not answer to the commercial request. CRG-TM_K.Brodzinski_2015.06.01

8. Repairs – preparation and comments • Integration of the activity in LS1 planning – M.Bernardini (EN-MEF), P.Cruikshank (VSC) and K.Brodzinski (CRG)Necessary to make it on CERN level for LS1 activity with regular weekly meetings – went very fine • Preparation of the cutting machine and necessary toolingSome difficulties to make the machine functioning and adapt it for needs of the repairs. This machine with associated equipment should be stored in traced place. Using during LS2 is not excluded. • Constitution of the repair team (CRG: 2 x 2 persons team + NV +KB)Very good repair team built, no problems with the activities running. • Repairs procedure including safety aspects issued and available via EDMSCan be used as guideline in the future if needed. CRG-TM_K.Brodzinski_2015.06.01

9. Repairs – schedule and execution Blue – new type of bellow ~8 weeks of preparation 26 weeks of repairs If necessary in the future, 1 single failure will need 2 weeks of preparation + 4 weeks of mechanical intervention + 2 weeks for re-pumping and validation CRG-TM_K.Brodzinski_2015.06.01

10. Repairs – photos Mockup b.276 CRG-TM_K.Brodzinski_2015.06.01

11. Repairs – photos CRG-TM_K.Brodzinski_2015.06.01

12. Lesson learned and conclusions • Significant increase of the number of compensators to be replaced and dedicated x-rays (16 bellows vs 1) after LS1 warm up -> required quick rescaling of the project and adaptation of budget, manpower and integration in the LHC planning with minor impact on LS1, what was achieved, • Deformation on the bellows with tightness lost can be seen on the vacuum insulation signal, deformations with conserved tightness can be seen by x-rays (no option to detect them with operation instruments), • In case of multiply bellows use it is recommended to order configuration with one ply tight and remaining plies vented to the insulation vacuum, • 1% of spares in our case was too small to cover sudden repairs needs, long delivery time was experienced from the supplier, • Tooling needed for the repairs (orbital cutting machine) to be stored properly in “ready for operation” condition, • Very good understanding and collaboration within CRG group and with external partners for tightness testing (TE-VSC) and planning (EN-MEF), • There is no need presently to replace any QRL compensator during LS2, • Work is documented and archived on EDMS Additional information: The total repairs, including new spares, did costs about 1 MCHF, AirLiquide delivered for free 2 He tracks (400 kCHF) as contribution to the repairs. Dedicated conference paper was published on ICEC25 by L. Tavian and K.Brodzinski CRG-TM_K.Brodzinski_2015.06.01

13. 4.5 K cold boxes repairs – introduction • There is 8 main cold boxes installed for the LHC cryogenics • 4 “new ones” with integrated LN2 pre-coolrer, ordered for LHC project • 4 upgraded ex-LEP with added LN2 pre-cooler for LHC 5 specific interventions were done during LS1 on 4.5 K cold boxes • LS1 planned work: • P8 QSRB Linde: Tightness problem on LN2 pre-cooler, leak from N2 circuit to insulation vacuum, pre-cooler out of service during Run1 • P18 QSRB AirLiquide: pollution of He circuit during using of the pre-cooler, pre-cooler used exceptionally in specific way during Run1 • P6 QSRB Linde: High helium residual signal in the cold box, cold box operational during LS1 with troubles of pumping (additionally external leak suspected) (this internal He leak could be treated as not planned work, since He signal increased during LS1 warm up – to be confirmed with APC) • LS1 not planned but done work – failures during LS1 transients: • P2 QSRA Air Liquide: He internal leak declared during sector warm up before LS1 (relatively fast transient applied) • P18 QSRB Air Liquide: He internal leak declared during start up of the pre-cooler for cool down of the sector (fast transient applied, urgent repairs action done) CRG-TM_K.Brodzinski_2015.06.01

14. General information • All interventions were performed according to rules applicable for confined space areas • Confined space supervised, leak detection and localization done by CRG-OA • Major of the repair interventions were done by external specialized firms with help from CRG-ME for the work preparation (opening/closing of the cold boxes, MLI removal/installation etc.) • Collaboration with many CERN external partners (TE-VSC, DGS-SEE, EN-MME) Michel, Krzysztof, Suite à notre visite de ce matin, même type de consignes que lors des précédentes interventions de ce genre, à savoir : •             Le tank est consigné électriquement (400v et 220v) ; •             Le process hélium est mis à la pression atmosphérique et l’installation mise en sécurité ; •             Le tampon principal et les trous d’hommes à proximité sont ouverts ; •             La zone autour du tank est balisée ; aucun travail en même temps, à côté ou au-dessus, pouvant ramener des vapeurs/gaz et/ou autres poussières à l’intérieur du tank ; •             Compte tenu de la configuration (1m de l’entrée correspondant au diamètre du tank), le contrôle du taux d’oxygène sera revu par vous-même avant d’entrer (pas besoin des pompiers pour la mesure, mais je les mets en copie pour juste pour info) ; •             Le superviseur prépare le permis d’entrer en espace confiné (joindre le présent email) ; •             L’intervenant doit porter un détecteur O2 ; •             Eclairage portatif en 24v ou frontale; •             Présence d’un surveillant (doté d’un téléphone portable) à l’extérieur du tank ; •             Mise en place d’un extracteur d’air pour le renouvellement à l’intérieur (soudage) ; •             Le matériels de soudage devra être relié au tableau/coffret via un transformateur de séparation de classe 2 (et restant à l’extérieur de l’enceinte) ; •             Pour le risque feu, les déchets de MLI devront être retirés (fait) et le MLI restant protégé pour couverture de protection feu ; •             Prévoir une couverture supplémentaire (à utiliser prioritairement) et un extincteur CO2 2 kg (uniquement si usage couverture insuffisant) ; •             Pour les fumées de soudure (corrosives et toxiques), on utilisera prioritairement : torche aspirante ou aspiration à la source – à défaut cagoule ventilée pour le soudeur ; •             Superviseur : un des superviseurs CERN formé (Liste TE/CRG jointe) ; •             Intervenants : Personnel Fives Cryogénie (pas de nécessité de suivre la formation dans ce contexte précis – à l’identique de leur précédente intervention en 2013) ; équipe CERN pour l’isolation ? •             Surveillant : un personnel CERN formé (idéalement le superviseur). J’ai bien noté  la préparation le lundi 11/08 PM, le début de l’intervention pour le mardi matin, et le passage d’Olivier Tison pour le contrôle Elec préalable. Cordialement, Olivier. Requirements for confined space intervention with welding, e-mail exchange with O. Prouteau CRG-TM_K.Brodzinski_2015.06.01

15. P8 QSRB – repairs (foreseen) • Leak from LN2 circuit to the insulation vacuum on He/LN2 HX • Cold box accessible via manhole • Repaired in situ by grinding and welding by FivesCryo • No special difficulties noticed CRG-TM_K.Brodzinski_2015.06.01

16. P18 QSRB – repairs (foreseen) • Leak from LN2 to He circuit on evaporator HX • Cold box accessible via end cap open • Repaired at CERN workshop by CRG-ME, with some difficulties with the first tentative but finally well done • Work preparation well anticipated • No special difficulties noticed, modifications applied on LN2 supply to reorient entering stream, Cu strip added for improvement of final LN2 evaporation CRG-TM_K.Brodzinski_2015.06.01

17. P6 QSRB – repairs (foreseen) • Leak from LP He line to the insulation vacuum on SS/AL transition of DN400 • Cold box accessible via main flange opening • Repaired in situ by Linde • Work preparation well anticipated • Difficulties: • to pass CERN qualification on AL welding, failed twice, no approved CERN procedure for that, when CRG asked MME for official statement on the welding qualification MME allowed for work … cinema! • Expensive repairs • Leak quite difficult to be localized, took relatively long time CRG-TM_K.Brodzinski_2015.06.01

18. P2 QSRA – repairs (not foreseen) • Leak from MP He line to the insulation vacuum on main He HX, appeared during thermal transient before LS1 • Cold box accessible via end cap open • Repaired in situ by FivesCryo • Work preparation well anticipated • Difficulties (first repairs): • Our electrical transformer not accepted by DGS-SEE – action blocked, other double insulated 400V electrical transformer not found at CERN, power generator rented for the intervention, then work done without special difficulties CRG-TM_K.Brodzinski_2015.06.01

19. P18 QSRB – repairs (not foreseen and urgent) • Leak on the pre-cooler on LN2/He plate HX from He line to the insulation vacuum • Cold box accessible via end cap open • Repaired in situ by FivesCryo • Work preparation – urgent! • Difficulties: • Work very urgent to allow for using of the pre-cooler for delayed cool down and to get the equipment repaired for Run2 • Limited availability of FivesCryo – thanks for help from CRG-ME to contact the firm for early intervention, • Developed special procedure to localize the leak • Our new electrical transformer not fully conform – work not blocked, transformer modified later CRG-TM_K.Brodzinski_2015.06.01

20. Lesson learned and conclusions • 5 repairs performed during LS1, each failure case was different (similar repairs at P8 and P18 on pre-cooler GHe/GN2 HX) – all leaks repaired and tight • Dedicated procedure defined and distributed for operation of the pre-cooler start up to protect the HX by minimizing thermo-mechanical stresses • Work globally well anticipated what was significant parameter contributing to the success of the repairs • Confined space course to be maintained as recommended for part of cryo operation team • Optimization but NO HURRY in the transients! 2 additional failures appeared due to fast transient operation (pressure from management and lack of imagination) – to be avoided in the future • Very good communication and understanding between the partners and external firms (with small exception for AL welder qualification) • Work is documented and is still to be properly archived on EDMS Many thanks to all involved people for their contribution in these important activities of LS1 ! CRG-TM_K.Brodzinski_2015.06.01