Operation of the LHC Cryogenics system and interface with beam & machine operation

1. Operation of the LHC Cryogenics system andinterface with beam & machine operation S. Claudet (CERN, Geneva) on behalf of the “Cryogenics Group” Technology Department

2. Outline • Introduction to LHC Cryogenics • Operation, organisation and results • Availability and interaction with beam operations • Summary LHC Cryogenics, interface with beam operation

3. LHC accelerator p-p collision 1034 cm-2.s-1, 14 TeV, 0.5 GJ stored energy Machine operation Technology 24 km of superconducting magnets @1.8 K, 8.33 T LHC Cryogenics, interface with beam operation

4. Layout of LHC cryogenics Magnets Distribution 8 x 18kW @ 4.5 K 1’800 sc magnets 24 km & 20 kW @ 1.8 K 36’000 t @ 1.9K 130 t He inventory LHC cryogenics is the largest, the longest and the most complex cryogenic system worldwide LHC Cryogenics, interface with beam operation

5. How does it compare ? Before LHC: existing experience for design, safety, controls, operation, availability, … ITER Tevatron, RHIC, Jlab, SNS, HERA, Tristan, … We did not start from scratch! LHC Cryogenics, interface with beam operation

6. LHC compressor station (x8) 4.2MW input power Bldg: 15m x 25m Oil/Helium Coolers Compressors Motors LHC Cryogenics, interface with beam operation

7. 18 kW @ 4.5 K Refrigerators (x8) 33 kW @ 50 K to 75 K - 23 kW @ 4.6 K to 20 K - 41 g/s liquefaction LHe: 3’600 l/h 4m diam, 20m long, 100tons LHC Cryogenics, interface with beam operation

8. Cold Compressor Active magnetic bearings 300 K under atmosphere 3-phase induction Electrical motor (rotational speed: 200 to 700 Hz) 200 to 800 Hz) Fixed-vane diffuser Outlet Cold under vacuum Spiral volute Axial-centrifugal Impeller (3D) Pressure ratio 2 to 4 Inlet 1.8K Units with cold compressors (x8) 125 g/s GHe from 15 mbar to P atm with 3 or 4 stages LHC Cryogenics, interface with beam operation

9. Electrical feed boxes for current leads 48 Boxes, 1200 leads LSSL2 of the LHC LHC Cryogenics, interface with beam operation

10. One LHC sector: production-distribution-magnets Total for 8 sectors: Compressors: 64 Turbines: 74 Cold Comp.: 28 Leads: 1’200 I/O signals: 60’000 PID loops: 4’000 • Extremely large installed cooling capacity • Complexity associated with 1.8K units • Extremely large distribution system => Recovery from failures can last from few minutes to 20 hrs, exceptionally 2-3 days x 13.5 From LHC Magnet String test 3.3km LHC Cryogenics, interface with beam operation

11. Interfaces: follow-up electrical perturbations EL perturbations and their impact on our LHC Cryo system Electrical systems recover in ms Cooling systems recover in min Cryo systems recover in hrs => A big incentive to be as tolerante to glitches as possible Duration [ms] Voltage change [%] Typical tolerance envelope LHC Cryogenics, interface with beam operation

12. Main reasons to superconducting For accelerators in high energy physics • Compactness through higher fields Ebeam ≈ 0.3 . B . r Ebeam ≈ E . L [Gev] [T] [m] [Gev] [MV/m] [m] Capital Cost Be sure that at design stage, working at higher temperature was considered, but not selected to maximise LHC beam energy => Cryogenic systems takes longer to recover from failures than conventional ones (but we work on it!) Operating Cost • Saving operating energy Electromagnets: Acceleration cavities Resistive: Pinput ≈ Ebeam Pinput ≈ Rs.L.E2/w Superconducting: Pinput ≈ Pref Rs ≈ RBCS + Ro RBCS ≈ (1/T) exp(-BTc/T) LHC Cryogenics, interface with beam operation

13. Interactions between LHC systems Powering OK orinterlock Static and Dynamic heat loads Beam related Dynamic heat loads LHC Cryogenics, interface with beam operation

14. Outline • Introduction to LHC Cryogenics • Operation, organisation and results • Availability and interaction with beam operations • Summary LHC Cryogenics, interface with beam operation

15. Key factors for operation On-call adapted LHC: A huge and complex system without significant buffer and frequent operator actions required Dedicated 24/7 required so far !!! • Equipment architecture: • Central liquefier to intermediate buffer, distribution decoupled • Cooling capacity production in line with demands • Type of operation • Transients (cool-down / warm-up) or various recovery • Alarm monitoring, simple reset actions, calling for experts • Detection of process degradation and curing action • HW checks and preventive treatment of slow evolving problems • Frequency of required actions: • Once per month, once per week • Once per 1-2 days LHC Cryogenics, interface with beam operation

16. Structure - Coordination - Outils Management Instrum-Cryolab Methods - Logistics Operation Accel. Operation Detect. Mechanics Electricity-Controls Coordinations: • Team Leaders + Management (1/wk) • Performance panel (1/2wks) • Operation / Maintenance panel • Methods & Tools panel Tools (web interface DB oracle): • e-logboog operation for any change of configuration (wanted or not) or observation and diagnostic request • Diagnostic tables, work-orders, intervention reports • Asset & spares management, intervention procedures • Maintenance plan • Scheduling LHC Cryogenics, interface with beam operation

17. Staff & team evolution People should be able to quit, newcomers should be integrated • High level requirements for recruitment (Bachelor & Masters) • Formalised induction process: Academic training - On the job training - Shadow shifts => Certification after ≈ 10 months as shift operator (alone!) • Senior operator (>3 yrs): Able with all sub-systems, ability to optimise production-needs-time • Certification diploma: Written - Site - Simulator - Improvement study (report + presentation) • If selected for indefinite contract: • Operation for 5 to 10 years • Ability to become “production Eng.” as site responsible • Ability to switch to support teams or another activity at Cern LHC Cryogenics, interface with beam operation

18. Cryo operator in Cern Central Control room Fixed displays Tendancy curves (summary) Process synoptics and orders Shift 24/7 LHC Cryogenics, interface with beam operation

19. Operation, indicators Alarms Powering Efficiency Global availability LHC Cryogenics, interface with beam operation

20. Outline • Introduction to LHC Cryogenics • Operation, organisation and results • Availability and interaction with beam operations • Summary LHC Cryogenics, interface with beam operation

21. Availability: a signal Yes/No is required CSCM Sum CM 8 sectors: Global availability T2  =   Achieved up time during required time / Required time  x 100 (operational availability)  T2 indicator w.r.t EN 15341 Cryo Maintain:Few important conditions checking integrity of HW, with slow power abort in case this signal is lost (leading to beam dump!) Cryo Start:set of conditions to allow powering of concerned sub-sector, with no action if powering started (illustrates good stability of process) CM CS CM SP set-point CS CM Possibility to treat thousands of channels in a structured way to match at best the LHC powering sub-sectorisation and the cryo sub-sectorisation LHC Cryogenics, interface with beam operation

22. LHCCryo global availability 2012 Target 2012 95% 2011 2010 2009 L7 TT891 • - Excellent 1st part to TechSTop • - Heavy works done during Technical Stop #1, and cabling weakness caused difficult recovery • Very moderate impact from High Luminosity operation in 2012 P4 CCs P18 oil P4 ice 2xP8 CCs SEU? P8+R5 LHC Cryogenics, interface with beam operation

23. Performance and origin of downtine Global availabilityas seen by LHC during beam operation periods Others according to relative ratio of their average for the 8 sectors Evolution: - 2010: Correcting early Cryo bugs - 2011: Adapting to SEU (corrected @Xmas) - 2012: So far rewarding !!! (260 days) (271 days) (137/290) (Full days, Mondays & Fridays of Technical Stops not counted here) LHC Cryogenics, interface with beam operation

24. Availability: from global to single plant Considering 8 independent sectors 2010-2011 Single sector and cryoplant LHC Cryogenics, interface with beam operation

25. Indicators: recovery categories & tendency From the books: Immediate effect of (good!) practice Annoying if frequent, to be kept low with moderate efforts Serious cases requiring specific monitoring and significant efforts + Better global control/tuning (operation, instrumentation) - ! SEU ! Nb stops • Less Cryo induced failures, . (but 3 VERY LONG ones!) - More Supply (EL) failures - ! SEU ! + No longer recurrent Cold Compressors particular issues (Leaks, electronics) Nice tendency, promising for 2012 or new surprises to come up? LHC Cryogenics, interface with beam operation

26. Operation structure & approach Control rooms: site - CCC- office • 2007/2008 cool-down & HWC: Per site, one experienced engineer with agreed minimum protocol to guide a local team of operators, with help of support teams (instrumentation, experts, controls) • Since 2009 and operation with beam: One operator in shift 24h/7d, more transverse structure site/CernControlCenter, procedures & operation tools • For machine controls (temperature, level, pressure): Basic interlocks and simple PID loops with generic tools for fast orders, now completed with automated sequences & procedures • Indicators: From temperature stability to daily availability on-line cool-down curves to on-line cryo-status LHC Cryogenics, interface with beam operation

27. Power Consumption for LHC Cryogenics Net gain ≈ 50 GW.h per year (3 MCHF / year !!!) Gain ≈ 8MW (20% ofinstalled power) Stop Cryoplant Installed power Operation with 8 plants Tests Power [MW] Cool down Cryo optimized power LHC physics 6200 h LHC physics 6500 h 700 h HWC 90% Cryo Availability 91% Cryo Availability 2009 2010 2011 1 3 5 Cryo unavailability breakdown 3 4 LHC Cryogenics, interface with beam operation 3

28. Helium inventory follow-up • Now < 30kg/day • With 50kg/day in 2011 and better control @Xmas, tendency to be confirmed • @50 CHF/kg - 50kg/day # 2’500 CHF/day - 150 t # 7.5 MCHF Helium invenrory [tons] LHC Cryogenics, interface with beam operation

29. Interfaces with Beam-OP Text zone • HW signals: • Cryo Start and Cryo Maintain towards Powering Interlock module • SW panels: • Cryo web page • People in Control Room (LHC): • 1 Eng in charge + 1 operator • 1 Cryo operator • 1 operator for technical infrastructure • Possible evolutions ? • Closer discussions with Eng. In charge in case of cryo problem • Other operators involved to help diagnostics/recovery • No longer cryo operators at night (on call only) LHC Cryogenics, interface with beam operation

30. Summary • LHC cryogenics is the largest, the longest and the most complex cryogenic system worldwide. We could achieve a reasonable availablity (> 90 %) so far with beams. This demonstrates that there are no big issues in concept, technology or global approach for operation. • Despite all our efforts, we had very hard time and lengthy commissioning to learn how to tune all these sub-systems together while permanently consolidating what was not conform. Experience has been converted into automatism, procedures, tools, training • Cryogenics operation is well integrated in central control room with LHC main systems, but operated/supported independently (about 50 people) • Maintenance is as well reaching an efficient preventive/corrective ratio, with efforts to be made for non-standard cases. We have to prepare for higher energies and intensities with continued gain in reliability ! LHC Cryogenics, interface with beam operation