Driving the LHCb Front-End Readout

1. Driving the LHCb Front-EndReadout • TFC Team: • Arek Chlopik, IPJ, Poland • Zbigniew Guzik, IPJ, Poland • Richard Jacobsson, CERN • Beat Jost, CERN

2. SWITCH SWITCH SWITCH SORTER SWITCH L1 FE SFC SFC L1 FE L1 FE L1 FE L1 FE L1 FE SFC SFC SFC SFC L1 FE L0 FE L0 FE ECAL RICH OT ST L0 FE L0 FE VELO L0 FE L0 FE MUON L0 TRIGGER HCAL L0 FE CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU SWITCH SWITCH READOUT NETWORK SWITCH SWITCH SWITCH SWITCH SWITCH STORAGE LHCb Readout Two levels of high-rate triggers Detector 40 MHz 1 MHz TFC SYSTEM Front-End electronics LHC CLK 40 KHz Event building For more details see Niko Neufeld’s talk in session T3 (RT-088) CPU farm

3. Physics trigger Local trigger (Optional) L0 / L1 L0 / L1 L0 / L1 Clock Clock Clock Clock Clock Readout Supervisor 1 Readout Supervisor 1 Single TFC master : Readout Supervisor Fan-out / Fan-in : Support partitioning Throttle Switch TFC Switch Readout Supervisor 2 Readout Supervisor 3 Readout Supervisor 4 VELO FE VELO FE ST FE OT FE RICH FE ECAL FE . . . Front-End TFC Architecture (1) Event building network

4. TFC Architecture (2) • TFC components • CERN Trigger, Timing and Control (TTC) distribution system. Common to all LHC experiments: • TTCtx (electrical-optical converters) • TTCoc (optical fan-outs) • TTCrx (Receiver chips) • - Components endemic to LHCb: • Readout Supervisors (“ODIN”) • TFC Switch (“THOR”) • Throttle Switches (“MUNIN”) • Throttle ORs

5. CERN TTC system 1 • Developed in the CERN RD12 project: Timing, Trigger and Control distribution system based on fiber optics • Transmitting two channels multiplexed: • A: Low latency 40 MHz signal • B: Two types of broadcasts with Hamming code protection: • Short broadcasts (8 bit data in 16 bit frame) • Long broadcasts (16 bit data in 42 bit frame)

6. CERN TTC system 2 • LHC: • Distribute LHC clock (40.08 MHz) and LHC orbit signal (11.246 kHz) to experiments over fiber with minimal jitter (~8ps RMS) • Experiments: • Distribute clock, trigger and control commands to the detector readout over fiber with minimal jitter Prevessin LHC Control Room Experimental hall several km

7. TTC in LHCb • Use of the TTC system in LHCb: • Channel A used to distribute (accept/reject signal) • L0 trigger (40 MHz --> 1.1 MHz accept rate) • Channel B used to distribute short broadcasts with encoded: • Bunch Counter Reset and L0 Event ID counter reset • L1 trigger (1.1 MHz --> 40 kHz accept rate) • Control commands (FE resets, calibration pulses) • Channel B used to distribute long broadcasts with: • IP/Ethernet destination address for the data transmission over the network to the CPU farm • Broadcast order on channel B is handled according to a priority scheme

8. L0 L1 Throttles LHC clock ECS - TTC encoding Trigger generator - Clock distribution ECS interface - L0 handling & distribution - L1 handling & distribution - Auto-trigger generator L1 broadcast generator - Trigger controller Trigger controller - Command broadcast generator Credit Card PC Cmd broadcast generator - ODIN Front-End - Exp. Control System interface Front-End TTC Encoder • Designed with emphasis on: • Versatile - to support many different types of running modes • Functions easily added and modified. Channel A/B DAQ Readout Supervisor “Odin” Experiment orchestra director - all mastership in a single module

9. Q_CMD Q_IOI Q_RND Q_L0 Q_PIPE TTC Q_GCS Q_T1B AFIFO TFIFO Q_L1 ODIN simulation • A lot of effort put on simulation. • Specs have been simulated in behavioral model with a behavioral model of the LHC machine, trigger system, and FE, using Visual HDL • Full Readout Supervisor with actual FPGA code and models of discrete logic has been simulated in the same simulation VisualHDL test bench • FPGA code also simulated at gate level in the same model

10. TFC Switch “THOR” Clock, trigger and command distribution and support partitioning • Crucial: Equal internal propagation delays. If skew too large, FE will suffer from timing alignment problems when using different Readout Supervisors. • Small jitter Pool of Readout Supervisors ECS MULTIPLEXERS ECS interface TTC information as TTC encoded electrical DELAYS V E L O S T O T R I C H . . .

11. Throttle Switch “Munin” Throttle feed-back from detector Front-End and trigger system to the appropriate Readout Supervisor Pool of Readout Supervisors ECS OR logic and history buffer ECS interface Throttle signals V E L O S T O T R I C H . . . Throttle OR module is only a variation of the same board as 32:1

12. Conclusions • TFC system architecture and use of TTC well establish • Different from the other LHC experiments with two levels of high-rate trigger • Emphasis on partitioning • Readout Supervisor • All mastership in one module • Provides a lot of flexibility and versatility • Switches • Partitioning to support testing, calibrating, and debugging well integrated • The first prototype of the Readout Supervisor “ODIN” built and tested • Final prototype of TFC Switch built and tested • Final complete prototype of Readout Supervisor ready for production • General purpose test board “FREJA” is being designed • Self-checking scan of the functionality by producing stimuli and receiving the output of the TFC system like a FE