90 likes | 239 Views
O 2 Project Online-Offline Computing. Pierre VANDE VYVRE. ALICE – TDR Run 3, 12 June 2013, CERN. Requirements and Strategy. Online System Requirements Sample full 50kHz Pb-Pb interaction rate (current limit at ~500Hz, factor 100 increase )
E N D
O2 Project Online-Offline Computing Pierre VANDE VYVRE ALICE – TDR Run 3, 12 June 2013, CERN
Requirements and Strategy P. Vande Vyvre • Online System Requirements • Sample full 50kHz Pb-Pb interaction rate (current limit at ~500Hz, factor 100 increase) • ~1.1 TByte/s detector readout and ~20 GByte/s to mass storage • Many physics probes: classical trigger/event filter approach not efficient • Massive data volume reduction • Data reduction by (partial) online reconstruction and compression • Store only reconstruction results, discard raw data • Demonstrated with TPC clustering since Pb-Pb 2011 • Optimized data structures for lossless compression • Algorithms designed to allow for offline reconstruction passes with improved calibrations • Implies much tighter coupling between online and offline reconstruction software
Hardware Architecture ~ 2500 links DAQ and HLT ComputingFarm Network ITS FLP 10 Gb/s 2 x 10 or 40Gb/s • 10 links at 10 Gb/s per FLP • ~ 100 Gb/s of I/O bandwidth TPC FLP FLP TRD EPN 10 or 40Gb/s FLP EMC PHO FLP DataStorage TOF FLP EPN Muon FLP L0L1 FTP FLP Trigger Detectors ~ 250 FLPs ~ 1250 EPNs First Level Processors Event Processingnodes P. Vande Vyvre
Detector – Online system interface - Standard detector-computing system interface - Each data block must be tagged with Trigger and/or Clock information P. Vande Vyvre
DDL evolution Det. Read-Out FPGA Det. Read-Out FPGA SIU IP CORE SIU IP CORE Run 1 LS1 Run 2 LS2 Run 3 P. Vande Vyvre
Custom DAQ I/O Run 1 LS1 Run 2 LS2 Run 3 P. Vande Vyvre
Commercial Network Interfaces P. Vande Vyvre • Generation of commercial network interface cards based on PCIe Gen 2 • Ethernet NIC: up to 4x10 GbEE.g. Chelsio • Infiniband HCA: up to 2x40 GbIB(QDR)E.g. Mellanox • Generation of commercial network interface cardsbased on PCIe Gen 3 (2013) • Ethernet NIC: up to 2x40 GbEE.g. Chelsio • Infiniband HCA: up to 2x56 GbIB (QDR)E.g. Mellanox • Strong motivation to adopt a commercial network for the DDL 3 !Existing network cards have an I/O bandwidth equivalent to what is neededfor Run 3
Detector – Online & DCSInterfaces DCSServer CTP DCS Network DCSServer Trigger & Busy Control FE Link DAQ/HLT FLP DAQ/HLT Network CRU DAQ/HLT EPN DDL3 RORC3 DDL3 SIU IP Front-End Data+Control Data Trigger+Data+Control P. Vande Vyvre
DDL versions over time P. Vande Vyvre Run 2: mix of DDLs 1 and 2 Run 3: probably a mix of DDL versionsStill many open points to be defined