ALICE Control System ready for LHC operation

1. ALICE Control Systemready for LHC operation ICALEPCS 16 Oct 2007 L.Jirdén On behalf of the ALICE Controls Team CERN Geneva

2. New ! New approaches or trends in experiment controls Outline • About ALICE • Tools & components • Typical applications • Data flow • User Interface • Integration, installation & commissioning • Conclusion

3. ALICE experiment 18 sub-detectors, 2 magnets 1000 members, 90 institutes, 30 countries Being installed & commissioned HMPID TOF center of Geneva LHCb ATLAS MUON - SPECTROMETER TRD CMS PMD ALICE SPD SDD SSD TPC PHOS Detectors not shown FMD , V0, T0, ZDC, EMC, CPV, ACO Introduction • ALICE Controls • Started 6 years ago • Small central team collaborating with • Detector groups & LHC experiments (JCOP) • In total ~100 people involved

4. ALICE Control Roomoperator consoles Counting roomscomputer farmsdatabases Experimental cavern Closed zone at runs radiationmagnetic field DetectorsStrong radiationstrong magnetic field ALICE environment ACR DAQ HLT Control ~ 50m racks detectors BEAM BEAM racks ~ 55m

5. Services Safety Gas DSS CSAM External Systems Electricity Cooling Accelerator Magnets Access Offline Infrastructure On-line B-field ECS Space Frame Trigger Beam Pipe DAQ Environment HLT Radiation SPD SDD SSD TPC TRD TOF HMP PHO FMD T0 V0 PMD MTI MTK ZDC ACO Controls context Controls Back-end 18 detectors ~150 sub-systems

6. New approach! FRAMEWORK TOOLS FINITE STATE MACHINE TOOLS ALARM ARCHIVAL DEVICE COMPONENTS Modeling operational behavior CONFIGURAT CONNECTION: OPC, DIM, DIP Integration of commonly used devices ACCESS TRENDING ALICE FRAMEWORK Standard communication & hardware access USER APPLICATIONS Tools and components SCADA SYSTEM PVSSII • Modular, distributed, • equipment oriented • error handling • file & Dbase access • graphics user interface

7. Time Projection Chamber TPC ALICE Detectors

8. TPC: 15 sub-systems ALICE: 150 sub-systems Jan 07: TPC being lowereddown into the cavern Gas (88 m3) Very High Voltage (100 kV) High Voltage (3 kV) Laser Low Voltage(60 kW) Mirrors Cameras Front-EndElectronics Cooling0.1 oC stability TPC sub-systems to control 510 cm

9. New ! Ethernet massively used as field-bus: - 1200 devices on private LAN - devices with Single Board Computers - large bandwidth required D Worker Node Worker Node worker node worker node Worker Node operator PVSS PVSS PVSS PVSS PVSS Operator Node DIM LV OPC DIM DIM HV OPC D PVSS D D Ethernet Ethernet LV Devices Systems layout ALICE Control Room counting room CANbus HV Devices Ethernet cavern DETECTOR FEE Devices

10. New ! • Much control software located on detectors - Previously in VME now in custom systems - Harsh environment & no access - redundancy – Control work done by electronics engineers OROC RO2 IROC RO1 4500 Front End Cards 0.6 million RO channels FEC FEC FEC FEC FEC FEC FPGA VLSI VLSI FPGA FPGA VLSI VLSI VLSI VLSI VLSI VLSI VLSI 128 ch 128 ch 128 ch DETECTOR TPC Readout Partition TPC Front-End Electronics Worker Node PVSS DIM 1 MB/sethernet OFF-DETECTOR ON-DETECTOR RCU Comm. Control Linux SBC Linux SBC MEMORY Board Control FPGA FPGA Bus Control Data bus I2C bus Front End Cards

11. OFF GO_STANDBY RESET CONFIG ERROR GO_OFF STAND BY Alarm GO_READY RAMP_UP RAMP_DW ON GO_STANDBY Operations layout TPC … DET RO 36x Control Units HV … FEE OROC LV IROC CH CH CH RCU CH CH RCU CH CH CH RCU CH Device Units CH CH Physicaldevices HV Devices LV Devices FEE Devices

12. operationsplane TPC DET RO operator operator PVSS HV D OROC FEE LV IROC CH CH CH CH CH CH CH RCU CH RCU CH CH RCU A systemsplane PVSS PVSS PVSS PVSS PVSS D D D D D ? commands DIM DIM LV OPC HV OPC DIM status, alarms B HV Devices LV Devices FEE Devices Operating principle • modular & flexible • operational aspects considered early & ‘built-in’ • easy partitioning

13. New approach! operationplane TPC DET RO operator operator PVSS HV D OROC FEE LV IROC CH CH CH CH CH CH CH RCU CH RCU CH CH RCU systemsplane PVSS PVSS PVSS PVSS PVSS D D D D D LV OPC HV OPC DIM DIM DIM HV Devices LV Devices FEE Devices Operating principle • modular & flexible • operational aspects considered early & ‘built-in’ • easy partitioning A commands status, alarms B

14. new trend ! Large amounts of data - more control channels - configurable electronics To Grid operator 100 000 change/s offline Conditions Physics Data processing archival “on change” From Detectors 6 Gbyte FEE FEE Main data flows Archival database LHC accelerator DAQ & Trigger Control SystemBack-end synchronization Technical services Configuration database 800 Single Board computers ~108 registers in FPGA’s, VLSI’s

15. ALICE User Interface Why needed? Small shift crew, changing often, non-experts Operating large set of sub-detectors GUI’s designed by each sub-detector Aim Same ‘Look and Feel’ Provide common utilities Login General utilities Controlzone Monitoringzone • ALICE UI • Fixed screen layout • System browser • General utilities Global Control Messenger Sub-sys status

16. ALICE CONTROLS USER INTERFACE Select HMPID ALARMS HELP PRINT PHONE LOGIN CONTROLS TOP-NODE PAGE DETECTOR STATUS HIERARCHICAL BROWSER MESSENGER SUB-SYSTEMS

17. ON D Worker Node Worker Node Worker Node worker node worker node operator PVSS PVSS PVSS PVSS PVSS Operator Node LV OPC DIM DIM HV OPC DIM central Operator Node D PVSS D D Ethernet CANbus Ethernet HV Devices Ethernet LV Devices TRD TOF DETECTOR FEE Devices 18 detectors, 150 applications Systems integration 100 PVSS systems:- a large large distributed system- of distributed systems TPC

18. ALICE ONLINE operator TPC ECS … DET RO 36x DAQ TRG HLT DCS LHC HV FSM top node … FEE OROC LV IROC CH CH CH RCU CH CH RCU CH CH CH RCU CH CH CH HV Devices LV Devices FEE Devices Detectors, infrastructure, services Operations integration FSM is an excellent tool for integration ! TPC

19. Status today • Installation • Controls Back-end is operational • 150 computers • Dedicated network for 1400 devices • User applications 90% ready, 50% installed • 150 detector applications • Originating from 20 groups in 30 countries • Conformity and quality control • Commissioning & operation • 13 of 18 detectors are installed being commissioned • 1st Cosmic run in December 2007 • 1st physics run in May 2008

20. New ! Conclusion • ALICE Controls System • is a new generation system • incorporates new innovative approaches • is being installed and is successfully used to commission the sub-detectors • fulfills all requirements • will be ready for operation with LHC in 2008

21. New ! New ! New ! New ! Posters • WPPB32 Cyber-security in ALICE • RPPB21 Finite State Machines for Integration and Control in ALICE • TPPA22 Standard Device Control via PVSS Object Libraries • RPPA36 Handling Large Amounts of Data in ALICE

22. Backup • ALICE UI • Fixed screen layout • System browser • General utilities

23. New ! Conclusion • ALICE Controls is a new generation system • Incorporates new innovative approaches • SCADA with common Framework • Massive use of Ethernet • Important “On-detector” controls • Handles large amounts of data • Operation based on Finite State Machines • Presently used for sub-detector commissioning • Fulfills all requirements • Will be ready for operation with LHC in 2008

24. parent commands states OFF GO_STANDBY RESET PVSS CU Behaviour (FSM) CONFIG operator operator ERROR access, ownership GO_OFF configuration archiving dB Partitioning Alarm handling STAND BY Alarm commands states GO_READY child child RAMP_UP RAMP_DW parent ON GO_STANDBY PVSS FSM interface access, ownership configuration archiving dB Partitioning device driver Alarms settings readings device Control & Device units

25. GAS BFI Data Interchange Protocol DIP ELE SFM Gas DSS CSAM COO BPI Electricity DIP Cooling MAG ENV DSS CSA Accelerator LHC Magnets ACC RAD DIP Detector Control Access OFF Offline Tools, components Back-End Systems B-field ECS ECS Computers Space Frame TRG Trigger Beam Pipe Networks DAQ DAQ Environment HLT HLT Radiation EMC ACO ZDC SPD SDD SSD TPC TRD TOF HMP PHO CPV FMD T0 V0 PMD MTR MTK SPD SDD SSD TPC TRD TOF HMP PHO FMD T0 V0 PMD MTR MTK ZDC EMC ACO CPV Integration Services Safety External Systems DCS Infrastructure On-line FSM top node 18 detectors ~150 sub-systems

26. New approaches in experiment controls • System based on SCADA and framework • Limited resources • Share developments between LHC experiments • Ethernet used as field bus • devices offer Ethernet interface increasingly • Becoming robust and less expensive • Large bandwidth needed • Important dataflow • Increasing number of control channels • Configurable electronics • Much controls located on the detector • Custom chips, single board computers • Operation based on Finite State Machine modeling • Operational needs prepared in the basic system

27. PVSS console ALICE CONTROL ROOM DIM client Front End Device (FED) Common interface? DIM server FEE common FEE specific 18 different FEE architectures interfaces MXI DDL PROFIBUS ETHERNET Etc. … MXI DDL PROFIBUS ETHERNET VME FEE H/W JTAG ON DETECTOR … SPD FEE HMPID FEE TPC FEE TRD FEE Front-End Electronics

28. Architecture of SPD FEE Control (PVSS) DIM Client Client Software Data, flags Commands Control and Monitoring Server DIM server CA1 Multithreadapplications CA2 MA1 MA2 Database VISA PCI-MXI-VME Control Agent reacts to commands received by server Monitoring Agent publishes data as DIM services Router JTAG Pixel Multi Chip Module