High Speed Data Receiver Card for Future Upgrade of Belle II DAQ

1. High Speed Data Receiver Cardfor Future Upgrade of Belle II DAQ Takeo Higuchi, High Energy Accelerator Research Organization (KEK) Nobu Katayama, Kavli IPMU, University of Tokyo Introduction – Belle II DAQ • Experimental apparatus parameters • Belle II detector and Belle II DAQ ~ The anticipated faint signatures of the new physics effect are detected with a large number of e⁺e⁻ collisions and a dedicatedly configured hermetic and granular detector. Near-detectorsignal digitizersand optical TX Far-site optical RX EVB (PC) COPPER readout system T. Higuchi et al.,IEEE Trans. Nucl. Sci. 52, 1912 (2005). ~700 RocketIOoptical links HSLB cards (x~700) COPPER boards (x~180) • x4 RX-card slots. • Pipeline buffer (4MB). • PrPMC slot for online processing. • Local and PCI buses bridged by PLX9054 bus-bridge chip. • x2 GbE data-output ports. HLT • Max link speed =3Gbps. • Virtex-5 FPGA. ~20MB/sper link at most ~⅓ ratereduction ~20m fibers Storage Opt TX Opt RX ~1.5GB/s New Data Receiving System The great success of the COPPER system: Block diagramof the new datareceiver card • The COPPER board can handle >30 kHz L1 trigger rate with a 90% dead-time reduction than Belle I proved in the Belle I DAQ operation. • The COPPER system is widely adopted in many HEP experiments. • Design concept of the COPPER board COPPER board (VME9U size) SFP+#2 DDR3 SFP+#1 Local bus PCI bus L1-asynchronous DAQ by FIFO Online data reduction by CPU .PrPMC Opt RX. FIFO Disadvantage of the COPPER system: Opt RX. FIFO • As the COPPER system embeds everything needed for the DAQ, the system became too complicated for the average Belle II users to start up and to maintain. MemoryI/FGenerator 9054 AURORA8b10b AURORA8b10b .GbE Opt RX. FIFO Opt RX. FIFO Simple-minded data receiver More use of commodities FIFO (1k x 288b) FIFO (1k x 288b) FIFO (1k x 288b) FIFO (1k x 288b) • A new data receiver card FIFO (1k x 328b) FIFO (1k x 256b) FIFO (1k x 4b) Design concept of the new receiver card New data receiver card (PCI Express) AURORAcontroller AURORAcontroller Statusindicators (LED) DDR3 controller DDR3memory(up to 4GB) x2 optical inputports (SFP+) x2 LVDS outputports (RJ45) DMAcontroller PCIecontroller Integratedblock Virtex-6 Use of commodities saves costs for software R&D and efforts for system maintenance. LVDS outputs to notify the DDR3 getting full. PCIe x4Gen2 Virtex-6(XC6VLX75T) PCIex4 Gen2 Performance Studies • New card’s performance study DELL 760MT DMA Core2 Duo 3.0GHzPCIe x16 4GB main memory CentOS 5.5 Readout PC New datareceiver card • Prototype study Corei7 3.4GHzx2 (PCIe x8 Gen3) 4GB main memory Scientific Linux 6.0 DMA Virtex-6 evaluationcard (ML605) PCIe x8 Gen1 Loop back Loop back FMC PCIe x4 Gen2 • Optical link +DMA performance †Throughput = 470MB/s Studies in cooperation with Sansei System Inc. • Optical link performance †BER < 1.8x10⁻¹³ • DMA performance †BER < 1.9x10⁻¹³Throughput = 635MB/s Mezzanine cardwith x4 SFP+ † 4MB per transaction Loop back DMA/PCIe 16kB image pre-stored in ML605 Opt TX 8b10b 8b10b Opt RX PC † Average over8192 transactions We continue system R&D with a milestone settled in the middle of 2017, when the peak luminosity will exceed 2x10³⁵ /cm²s.