Overview of LHCb Electronics installation aspects V. Bobillier; L. Roy; J. Christiansen

1. Overview of LHCb Electronics installation aspects V. Bobillier; L. Roy; J. Christiansen CERN - LHCb V. Bobillier

2. LHCb experimental cavern • Racks and rack control and monitoring system • Detector safety system and its implementation in LHCb • Mains power distribution for the experiment • Low voltage power supplies choices • Cabling constraints and installation • Optical fiber cables • Global grounding in cavern • Status of installation V. Bobillier

3. LHCb • The experimental cavern Protective shielding wall Detector • Counting house • 3 levels • 150 racks V. Bobillier

4. LHCb • The experimental cavern Status in June 06 V. Bobillier

5. Rack types • Front-end electronics racks (~50): standard type for crates and electronics boards (vertical recirculating air flow) • Air circulated by turbine units • Heat exchanger cool air • Fan trays distribute air homogeneously • PC farm racks (~80): Special custom made cooling rear door (horizontal open air flow) • Fresh air taken externally from front of rack • PC cooled down by air • Air is cooled down before being extracted in rear of rack Side view of PC racks Front view of front-end electronics racks V. Bobillier

6. Rack control and monitoring system • In every Front-end electronics racks. • Monitors: • Temperature • Humidity • Smoke detection (DSS only) • Acts on (via ECS or DSS in emergency cases): • Mains electrical power distribution for a group of racks. • Gives alarms (possibility to act remotely to trigger fire extinguisher systems). • PC racks: Simplified system based on same principle is implemented (temperature monitoring made via PCs; Fans monitored via ELMBs (ECS); smoke detector and fire extinguishing system for entire counting house). V. Bobillier

7. DSS (detector safety system) • Protects equipment from being destroyed when conditions are out of standards. • Independent from other systems, redundant, has its own UPS and is backup supplied by Diesel generator. System has been made in collaboration with IT/CO. • Decision matrix in the central PLC takes appropriate actions. • Linked to ECS (Experiment Control System) and CSS (CERN Safety System). • System consists of simple temperature, humidity, water leak sensors and outputs (for actions). • Principal actions are: • Cut mains power • Close cooling water valves • Turn HV off • Trigger fire extinguishers (not automated) • Water mist for CPU farm • CO2 for F-E racks V. Bobillier

8. Mains electrical power • Electrical distribution principle for the experiment: • 2 sets of 1.25 MVA transformers and distribution panels: Separation between computing farm and front-end electronics. • Distribution to racks via 2 levels of distribution panels • Local PLC system for supervision (monitoring and basic control on/off). • Main components of DAQ (main switch; control PCs; readout supervisor) backup supplied by UPS (160 kW). x 2 Supervision system V. Bobillier

9. Mains electrical power • Problems foreseen: • Harmonics produced by PC equipments (poor PFC) •  Solution: split distribution in 2 (2 TFOs) • Important inrush currents (no soft start) mainly in PCs •  Solution: temporization relays in distribution boxes in addition to adapted circuit breaker curves (not sufficient). • Measurements performed by CERN electrical support group • Measurements of the rack mounted PCs for the LHC experimentshttps://edms.cern.ch/file/503582/1/second_meas_report_rackmounted_PC_final1.pdf • Measurements of the electronics crates for LHC experimentshttps://edms.cern.ch/file/442186/1/Crates_LHC_experiments_rev2.pdf Crate LVPS THD (total harmonic dose) measurements 300 A PC rack inrush current PC rack THD measurements V. Bobillier

10. Mains electrical power • In rack distribution box Position of the 19” electrical distribution box in the electronics racks Circuit breakers and adjustable temporization relay Secondary distribution panels V. Bobillier

11. Low voltage power supplies • LVPS in the cavern: • From radiation simulations (factor 2 included), worst location : • Radiation total dose over 10 years: 5.8*103 Rad • 1MeV neutrons/cm2 over 10 years: 1.9*1012 • Above 20MeV hadrons/cm2 over 10 years: 5.5*1010 • Magnetic field always below 200 Gauss. • Qualification : • Radiation total dose: 14*103 Rad • 1MeV neutrons/cm2: 1*1012 • Above 20MeV hadrons/cm2: 2*1011 V. Bobillier

12. Low voltage power supplies System (~600 channels in total): 380 Vdc • Advantages: • AC/DC converter and monitoring/control of power supply protected from radiation and magnetic field by the shielding wall. • Smaller length of large diameter low voltage power cables (about 20 meters). • Less dissipated power in cables (~36%). • Better voltage stability. • All F-E have local linear regulators • Total cross section of the cables going from the AC/DC converter to the power supply (DC/DC) reduced by ~ factor 10. • Disadvantages: • Limited accessibility to the power supplies (DC/DC part). Question of reliability… • Basic on/off control and voltage and current monitoring functions per channel. • Power supplies (DC/DC) converter must stand radiation. V. Bobillier

13. Cabling Counting house Sub-detectors Cables have to run from sub-detectors (patch panels and on-detector front-end electronics racks) to counting house racks. V. Bobillier

14. Cabling Cable routes V. Bobillier

15. Cabling Cable trays integration In front of the counting house cables must be correctly distributed to enter in the right rack column. Philippe Preau V. Bobillier

16. Cabling Chicane to go through shielding wall Reduced space for cables in chicane. Only 350 * 2000 mm per side. Side view Top view V. Bobillier

17. Optical fiber links • In LHCb more than 7000 optical fibers for readout • Total bandwidth ~11,2 Tbit/s • Assuming BER at 10-13 errors/bit results in ~1 error/s •  Extensive tests needed. Particular care must be taken during and after installation. • BER measurements performed with 6dB additional attenuation in optical links. • Total attenuation loss measurements done after installation. Multi-ribbon optical fiber trunk cable (96 fibers, 8 ribbons of 12 fibers, MTP/MPO connectors) V. Bobillier

18. Optical fiber links • Partial view of a sub-detector requirements for links between cavern and CH ~ 300 units V. Bobillier

19. Grounding • Global grounding principle in cavern: • In addition of safety earth connections, inter-connections are made to all metallic structures (supporting rails, platforms, etc.) • 120mm2 ground conductor running along every 4th cable tray. • Goal: Force ground loop currents circulating in low impedance grounding network. • Problem encountered: Connections with iron reinforcement in concrete…? V. Bobillier

20. Grounding Grounding of racks in counting house: Grounding of the racks. • Metallic structure in false floor of barrack is connected in many points to cooling pipes, cable trays and earth cables running under racks. • Every CH level is connected to earth in the cavern. • All racks are connected via a connecting braid to this mesh. • In addition: • All copper cables (LV, HV and ctrl) are shielded. • Cables are installed in metallic closed (covered) cable ducts. V. Bobillier

21. Status • Rack control and monitoring: All racks ready. System is installed everywhere. Commissioning will be done soon. • DSS: DSS is running (general purpose sensors connected, interlocks to mains power network and water valves installed and tested). Sensors for sub-detectors defined to be installed. • Mains electrical power: Power available in all racks. PLC supervision system still to be commissioned. • LVPS: Waiting for final deliveries. • Cabling: Copper cables: 70% pulled. Connector installation will start in coming weeks. Optical fiber cable pulling will start in November. • Grounding: In counting house (racks): done. In cavern: still some interconnections to be made. V. Bobillier

22. Questions / Comments V. Bobillier