1 / 13

A Scalable 1 MHz Trigger Farm Prototype with Event-Coherent DMA Input

This paper presents a scalable 1 MHz trigger farm prototype with event-coherent DMA input. The prototype uses a 3D torus topology and TagNet for data transfer. The paper discusses the trigger concept, simulation, and algorithm, as well as the performance and cost of the prototype. The team involved in the project includes researchers from Heidelberg, CERN, and Dubna.

mikedunn
Download Presentation

A Scalable 1 MHz Trigger Farm Prototype with Event-Coherent DMA Input

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Scalable 1 MHz Trigger Farm Prototype with Event-Coherent DMA Input V. Lindenstruth, D. Atanasov, I. Kisel, A. Walsch ( KIP, Uni-Heidelberg, Germany ) H. Muller, D. Altmann, A. Guirao, F. Vinci dos Santos ( CERN, Geneva, Switzerland ) • LHCb Level-1 Trigger • Trigger Concept • Trigger Prototype • Trigger Simulation • Trigger Algorithm Ivan Kisel, Uni-Heidelberg, RT2003

  2. e h  Level-1 Trigger for LHCb TT VELO (T1—3) • Find VELO 2D tracks and reconstruct 3D primary vertex • Reconstruct high-impact parameter tracks in 3D • Extrapolate to TT through small magnetic field  PT • Match tracks to L0 muon objects  PT and PID • Select B–events using impact parameter and PT information • Use T1—3 data to improve further selection (5—10% of events) Ivan Kisel, Uni-Heidelberg, RT2003

  3. Trigger Concept Reduce rate from 1 MHz to 40 kHz • Send data to RU(8 kB/evt  8 GB/s): • VELO + TT + L0DU • T1—3 (~5—10% on CPU demand) • Traffic shaping --> use Scheduler ! • NIC with Remote Direct Memory Access ! • Prototype: 2D torus with 32 dual nodes at 1.24 MHz • Trigger farm : 3D torus with up to 1200 CPUs X->Y routing PC farm (2D torus) TagNet x Scheduler y Data - RU - CN Ivan Kisel, Uni-Heidelberg, RT2003

  4. Tag Output Stage Tag Input Stage Scheduler Core List of free computing nodes Control Register Status Register Scheduler - Basic Block Diagram Free CN IDs entries • The supervisor of the system. • Handle a coherent data transfer between RUs and CNs. • Feed the TagNet with tags for synchronous data transfer in the RUs. TagNet Feed Event entries TagNet Feedback User Control Ivan Kisel, Uni-Heidelberg, RT2003

  5. Readout Unit (RU) Command Tag Message Tag Tag In Tag Out Tag Buffer Message Execution C/M ? MUX Command Execution Data In Subevent Buffer DMA NIC Out NIC In PCI Bus NIC Interface Ivan Kisel, Uni-Heidelberg, RT2003

  6. Trigger Farm Prototype in Heidelberg 1 year 64 CPUs 6 Gbit/s NIC 2D torus >1 MHz 480 MB/s p-p 450 MB/s x-y Ivan Kisel, Uni-Heidelberg, RT2003

  7. GUI of Prototype • Automatic setup of the compute farm • Configure and control processes on every CN Ivan Kisel, Uni-Heidelberg, RT2003

  8. 3D Torus Topology 4x4x(1+2+1) 3D Core 1D Cover TagNet Scheduler Data z x y • TagNet – schedule and send small data packets • Core network – distribute data to the target compute nodes • Cover network – increase number of compute nodes • X->Y->Z routing path - RU - CN Ivan Kisel, Uni-Heidelberg, RT2003

  9. Ptolemy II Simulation of the Trigger 3D Torus (6x6x8)275 CNs Ivan Kisel, Uni-Heidelberg, RT2003

  10. Simulation of the Trigger --- Results 2.1 MHz measured ! 128 B/RU Fast Compact 1200 CPU Fast Response VELO Scalable +5% T1--3 Scheduler VELO events Z VELO T1-3 New VELO Ivan Kisel, Uni-Heidelberg, RT2003

  11. Tracking Efficiency and PV Resolution Track subsets 2D % 3D <70> <8> Reference B long Reference prim. long Reference B Reference primary Reference set All set Extra set Clone Ghost 97.7 99.1 96.6 98.7 97.0 93.6 81.1 4.5 6.3 95.1 97.1 93.3 93.9 92.3 87.5 70.2 4.0 9.3 X/Ycore17 m Zcore46 m Ivan Kisel, Uni-Heidelberg, RT2003

  12. Trigger Performance 1) Tracking efficiency 97—99% 2) PV resolution 46 mm 3) Timing 4.8 ms CPU Expect a factor 7—8 in CPU power in 2007 (PASTA report) => we are already within 1 ms ! 4.8 ms  Events 17 ms FPGA co-processor  time (ms) Mean: 15 ms  Events • Cellular Automaton algorithm • FPGA co-processor at 50 MHz • 8 processing units running in parallel • => 15 ms ! Max: ~130 ms  time (ms) Ivan Kisel, Uni-Heidelberg, RT2003

  13. Summary: • Demonstrated Architecture with 3D torus and TagNet • Prototype of 64 CPUs has shown stable work at > 1 MHz • The Simulation is based on the prototype measurements • The Algorithm has high performance on tracks and vertices • The Cost is 1300 kCHF (500 CPU) / 2300 kCHF (1000 CPU) • The Team --- Heidelberg, CERN and Dubna Ivan Kisel, Uni-Heidelberg, RT2003

More Related