50 likes | 152 Views
LKr Readout with CARE modules. CARE Features. Each CARE consists of 4 sectors → total 896 “virtual modules” in LKr. The analog LKr signals are digitized with 40 MHz. Raw data input is 8x120=960 MB/s per LKr sector (16 chans).
E N D
LKr Readout with CARE modules NA62 TDAQ WG Meeting, Anacapri, 2 Sep 2009
CARE Features • Each CARE consists of 4 sectors→ total 896 “virtual modules” in LKr. • The analog LKr signals are digitized with 40 MHz. • Raw data input is 8x120=960 MB/s per LKr sector (16 chans). • Each LKr sector (8x CPDAS = 16 chans) has two DDR2 SODIMM socketswith maximum 2 x 4 = 8 GB (intermediate) data storage. • This corresponds to ~8.3 s of continuous data-taking @ 40MHz. • Read-out is with 1 GbE link/sector = ~80 MB/s using jumbo frames(4 GbE links per CARE), which represents approx. 1 MHz event rate maximum (compared to 10 KHz for the CPD) if readout is sustained between spills. • The same physical format as the CPD in order to re-use the CPDTR trigger sum board, the 7000 CPDAS cards and the FASTBUS crates for power and cooling • CARE has no FASTBUS interface. • 80 MHz Clock as CPD, with SoB (Start of Burst) and EoB (End of Burst)and other functionalities (to be defined) by addition of a TTCrx device. • CAN-bus for the DCS (Detector Control System)which interfaces to an ELMB module on the CAREfor configuration & monitoring of the CARE module. NA62 TDAQ WG Meeting, Anacapri, 2 Sep 2009
LKr Readout with CARE (1) • First attempt at defining an architecture • 224 CAREs = 896 LKr 16-chan sectors, continuous (untriggered) data-taking (960 MB/s per sector) in burst • Start-of-burst signal absolutely necessary to sync all CARE modules • A (bidirectional) GbE connection/sector (max 80 MB/s).. • ..connecting via GbE switches.. • ..to processor nodes that doevent building and processing(e.g. zero-suppression) andforward the events to NA62 DAQ In addition: • Analog trigger sums go tothe LKr L0 trigger sysstemusing the existing cabling • Several CAN buses (e.g. 8,i.e. 28 CAREs/bus) forconfiguration & monitoring NA62 TDAQ WG Meeting, Anacapri, 2 Sep 2009
LKr Readout with CARE (2) Readout Controller • central role in LKr readoutsteered by inputs from Lx trigger(s) • issues broadcasts to CAREs, containing 40+ event requestsand the address of a singledestination eventbuilder/processor node port • event request = timestampof the event from the start-of-burst • each processor node hasmultiple GbE ports (e.g. 4) • each CARE assembles and sends the data of the 40+ events in one (jumbo) frame (ca. 9000 bytes) to the destination node port (jumbo in order to optimize use of GbE bandwidth) • note: Ethernet between RC+CARE+ switches+nodes and TCP/IP between nodes+TDAQ NA62 TDAQ WG Meeting, Anacapri, 2 Sep 2009
LKr Readout with CARE (3) GbE Switches • CARE sector GbE output @ 80MB/s→ (80M/9000)*47 = ca. 400 kHz evtrate • event building: 896 jumbo frames(47 evts, ca. 8 MB) go to one output port(or to 4 ports of one node in parallel?) • at 80MB/s (max GbE) this takes 100 msnot incl. latency • one port then is able to consume470 evts/s • so to sustain 400 kHz event rateneed 400k/470 = ca. 850 GbE ports= ca. 200 processor nodes ifthere are 4 ports/node NA62 TDAQ WG Meeting, Anacapri, 2 Sep 2009