410 likes | 524 Views
Status of the ALICE DAQ. T. Anti č i ć. LHC days in Split -2006. ALICE DAQ. Flexibility. 17 detectors Pb-Pb pp and pA. Synchronized / standalone tuning. Big events (86.5 MB) Large bandwidth (1.25 GB/s to tape) Low interaction rate (10 KHz) Complex triggers
E N D
Status of the ALICE DAQ T. Antičić LHC days in Split -2006
ALICE DAQ Flexibility • 17 detectors • Pb-Pb • pp and pA • Synchronized / standalone tuning • Big events (86.5 MB) • Large bandwidth (1.25 GB/s to tape) • Low interaction rate (10 KHz) • Complex triggers • Different sets of detectors • Small events (2.5 MB) • High interaction rate (200 KHz) • Less bandwidth • Simpler triggers 25 GB/s raw from the detectors => 1.25 GB/s data files tome.anticic@cern.ch LHC days in Split -2006
TDS TDS ALICE DAQ Architecture Rare/All CTP BUSY BUSY LTU LTU DDL H-RORC HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Event Fragment 163 DDL 343 DDL 10 DDL 10 D-RORC 10 HLT LDC 425 D-RORC LDC LDC LDC LDC LDC 200 Detector LDC Load Bal. Sub event Event-Building Network EDM Event 50 GDC 10 DS GDC GDC GDC DS DS GDC File PDS Storage Network 25 TDS tome.anticic@cern.ch LHC days in Split -2006
TDS TDS ALICE DAQ Architecture Trigger High-Level Trigger CTP LTU LTU HLT Farm TTC TTC FEP FEP Detectors FERO FERO FERO FERO LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network Data Acquisition tome.anticic@cern.ch LHC days in Split -2006
CTP LTU LTU TTC TTC FERO FERO FERO FERO TDS TDS Trigger Initiates detector readout of selected events Rare/All BUSY BUSY HLT Farm FEP FEP LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
CTP LTU LTU TTC TTC FERO FERO FERO FERO TDS TDS Trigger Central Trigger Processor: decisions Rare/All BUSY BUSY HLT Farm FEP FEP LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
CTP LTU LTU TTC TTC FERO FERO FERO FERO TDS TDS Trigger Local Trigger Unit: receives decisions for each detector Rare/All BUSY BUSY HLT Farm FEP FEP LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
CTP LTU LTU TTC TTC FERO FERO FERO FERO TDS TDS Trigger Timing, Trigger and Control system: broadcast toelectronics Rare/All BUSY BUSY HLT Farm FEP FEP LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
CTP LTU LTU TTC TTC FERO FERO FERO FERO TDS TDS Trigger Front-End Readout Electronics: use triggers, signal BUSY Rare/All BUSY BUSY HLT Farm FEP FEP LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
CTP LTU LTU TTC TTC FERO FERO FERO FERO LDC TDS TDS Trigger Some DAQ components can feed back information Rare/All BUSY BUSY HLT Farm FEP FEP LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • Data-flow from electronics to permanent storage CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Event Fragment 163 DDL 343 DDL 425 D-RORC 200 Detector LDC Load Bal. Sub event Event 50 GDC 10 DS File PDS 25 TDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • Detector Data Links: optical transmission CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Event Fragment 163 DDL 343 DDL PDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • DAQ Read-Out Receiver Card: PCI interface CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Event Fragment 425 D-RORC PDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • Local Data Concentrators: ship sub-events CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Event Fragment 200 Detector LDC Sub event PDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • Global Data Collectors: build events CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Sub event Event 50 GDC PDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • Event Distribution Manager: load balancing CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Load Bal. EDM PDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • Transient Data Storage: local storage before migration CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO File PDS 25 TDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • Permanent Data Storage: file archive CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO PDS tome.anticic@cern.ch LHC days in Split -2006
LDC LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC Storage Network TDS TDS Data Acquisition • DAQ Services: database and central facilities CTP LTU LTU HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO 10 DS PDS tome.anticic@cern.ch LHC days in Split -2006
HLT Farm FEP FEP LDC LDC TDS TDS High-Level Trigger • Filter events to optimize amount of valuable data CTP LTU LTU DDL H-RORC TTC TTC FERO FERO FERO FERO 343 DDL 10 DDL 10 D-RORC 10 HLT LDC LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
HLT Farm FEP FEP LDC LDC TDS TDS High-Level Trigger • Detector Data-Links: transfer of event fragments CTP LTU LTU DDL TTC TTC FERO FERO FERO FERO 343 DDL LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
HLT Farm FEP FEP LDC LDC TDS TDS High-Level Trigger • HLT Read-Out Receiver Card: data reception CTP LTU LTU H-RORC TTC TTC FERO FERO FERO FERO LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
HLT Farm FEP FEP LDC LDC TDS TDS High-Level Trigger • Front-End Processors: data analyzing and decision CTP LTU LTU TTC TTC FERO FERO FERO FERO LDC LDC LDC Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
HLT Farm FEP FEP LDC LDC TDS TDS High-Level Trigger • DDLs, D-RORCs & LDCs: injection of results in DAQ CTP LTU LTU TTC TTC FERO FERO FERO FERO 10 DDL 10 D-RORC 10 HLT LDC LDC LDC LDC HLT decision Event-Building Network EDM GDC GDC GDC DS DS GDC PDS Storage Network tome.anticic@cern.ch LHC days in Split -2006
Implementation • Transfer of data to the DAQ • Computers • Network • Infrastructure • Software tome.anticic@cern.ch LHC days in Split -2006
Detector Data Link Transfer of data to the DAQ • Radiation tolerant SIU (Source Interface Unit) • D-RORC • DDG: DDL Data Generator (for tests) tome.anticic@cern.ch LHC days in Split -2006
Installation Pre-installation of optical fibres finished in the backframe tome.anticic@cern.ch LHC days in Split -2006
Test setup DDL, DDG, Trigger TRG2 LTU TRG1 DDGs DDG1 DDG2 DDL2 DRORC D-RORC DDL1 LDC Trigger crate tome.anticic@cern.ch LHC days in Split -2006
Computer choices • Intel 32 (LDC,GDC) • AMD 64 (DS) • Scientific Linux CERN – SLC3 • GDC: CPU/memory performance • LDC: number / architecture of PCI slots • DS: database, multithreading performance SLC4 when certified tome.anticic@cern.ch LHC days in Split -2006
Event Building Switch • ALICE baseline: • TCP/IP over switched Ethernet • Event-Building Switch • CERN frame contracts • Needs of IT and experiments • Good prices • On site maintenance (company+IT) • Switch qualification test • Special set-up (IT): 60 PCs • DATE software used for benchmarking • Switch selected: Force 10 Model 1200Delivered and installed tome.anticic@cern.ch LHC days in Split -2006
Experimental area Computing Center DAQ 10Gb Ethernet Counting rooms DDLs Machine: LHC @ CERN, Point 2 Cavern tome.anticic@cern.ch LHC days in Split -2006
DAQ Counting Room • Tight space: 70m2 • 33 racks • ~300 computers tome.anticic@cern.ch LHC days in Split -2006
ALICE DAQ @ P2 PX24/CR1 DDL patch panels NetworkSwitch PDU LDCs GDC Storage Clean room SXL2 Server NetworkGateway tome.anticic@cern.ch LHC days in Split -2006
ALICE Control Room • 30 workstations • separate room for PCs • console access to • machines in • all counting rooms • Multiscreen PCs • Nvidia NVS 280 • Linux Xinerama tome.anticic@cern.ch LHC days in Split -2006
DAQ Software DAQ software • DDL software • DAQ framework (DATE V5) • Performance Monitoring (AFFAIR) • Data quality monitoring (MOOD) Auxiliary Software: • DIM, SMI, MySQL, Tcl/Tk, libshift, ROOT, VMEbus driver Information dissemination • Documentation: 4 DAQ sw packages + ECS: fully documented.User’s guide released and printed • Regular DAQ trainings (70 people in last 3 years) Linux • Linux SLC3 now. • ALICE DAQ contributing to Linux SLC4 certification.Transition to SLC4 scheduled before end ’06. Testing: • reference system, on-line data challenges, test beams tome.anticic@cern.ch LHC days in Split -2006
DATE Run Control • Control • Configuration • Main parameters • Display • SMI State of all nodes • Main counters of all nodes • Compatible with ECS tome.anticic@cern.ch LHC days in Split -2006
DATE Configuration Database • Database content • DATE RolesActors of DATE system:LDCs, GDCs… • TriggerTrigger masks • DetectorsFront-end equipment of LDCs • Event building controlEvent building rules • BanksMemory banks to operate DATE • Released in DATE V5 tome.anticic@cern.ch LHC days in Split -2006
DATE Infologger • Online view • Online selection of visible fields • Online selection of message displayed tome.anticic@cern.ch LHC days in Split -2006
Data quality monitoring: MOOD • MOOD: Monitoring Of Online Data • DATE + ROOT environments • MOOD framework • Interfaces to detector code • Applications: • Raw data integrity • Detector performance tome.anticic@cern.ch LHC days in Split -2006
Performance monitoring: AFFAIR • Individual nodes view • CPU usage • Input/Ouput • Node status • System view / Aggregated performances • Event building bandwidth • Event numbers tome.anticic@cern.ch LHC days in Split -2006
Conclusion • Data transfer • DDL and D-RORC being produced • DAQ software ready • Software (DDL sw, DATE, AFFAIR, MOOD) released (rpm) and documented. • satisfies the functional + performance requirements • runs on single machine and large-scale setups • Linux SLC3 now. Transition to SLC4 scheduled before end ’06. • is tested at reference system, data challenges, test beams • is ready for the test and commissioning phase • DAQ fabric hardware • Event building switch qualified and being installed • PCs and storage selected. Tenders include most recent equipment. • Installation • Services for DAQ finished • Optical fibres progressing with detector installation tome.anticic@cern.ch LHC days in Split -2006