1 / 42

Readout Electronics for the SuperKEKB

Readout Electronics for the SuperKEKB. Contents. Introduction Conceptual design The COPPER-II Readout Scheme COPPER-II Performance Schedule Summary. Requirements for Readout Electronics. SuperKEKB. Detector. Trigger rate > 10 kHz Data size ~ 300 kB/ev Readout ~ 50 k channels.

gschafer
Download Presentation

Readout Electronics for the SuperKEKB

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Readout Electronicsfor the SuperKEKB Super B Factory Workshop

  2. Contents • Introduction • Conceptual design • The COPPER-II • Readout Scheme • COPPER-II Performance • Schedule • Summary

  3. Requirements for Readout Electronics SuperKEKB Detector • Trigger rate • > 10 kHz • Data size • ~ 300 kB/ev • Readout • ~ 50 k channels TriggerSystem ReadoutElec. EventBuilder Storage DAQ Also good for J-PARC, K2K …

  4. Conceptual Design What are keys in designing the readout system?

  5. Event buffers for pipelined DAQ Data size reduction by processor on data path System Design from detector Data path Digitization Readout FIFO Processor Consequent design of readout electronics to EB

  6. Design from Cost View • Flexible system • Common platform + user-defined ADC/TDC. • Implementation of ADC/TDC and CPU as modules. • Compact design • Less #-of-crates to house platform boards. • Smaller room of electronics hut. • Commodity usage • Good products at low price in computer market. • Easy to follow/import newest technology.

  7. ADC/TDC Module ADC/TDC Module ADC/TDC Module ADC/TDC Module Design Overview mezzanine(add-on) module FIFO PMC PCI bus local bus Detector signals PMC CPU Network IF Bridge Trigger Module

  8. Conceptual Design (Front-end I/F) flexibility • User-defined ADC/TDC module compactness Add-on type Module Readout FIFO Detector signal L1 triggerpipeline ADC/TDC 0xaa55 0x0246 0xf3b7 … sampling clock trigger signal

  9. Conceptual Design (L1 Trigger Pipeline) pipeline Digitized data in L1 trigger Ready switch EMPTY Data READ Data READ Data READ Trigger HOLD Always WRITE Always WRITE switch Data out (to Readout FIFO) busy

  10. Conceptual Design (FIFO) pipeline Local bus Detector ADC/TDC Module DMA Detector ADC/TDC Module PCI bus Bridge Detector ADC/TDC Module 32 Detector ADC/TDC Module trigger signal

  11. Conceptual Design (Back-end) online data reduction • Data path = PCI bus • PCI mezzanine modules (PMC) flexibility compactness Data suppression commodity use Local bus PCI = Up to 133 MB/s CPU EB DMA Bridge Network PCI bus Trigger trigger signal

  12. Conceptual Design (Processor) flexibility • Modular CPU • Modularity = easy to upgrade. • Intel x86 series: most familiar architecture. • Operating System = Linux • Easy to develop device driver / software. • Low price. commodity use

  13. What’s ‘PMC’?- PCI Mezzanine Card Standard 74×149 mm2 PMC Ethernet card PCI Ethernet card • PMC is 100% compliant with the PCI. • PMC is suitable for high density applications. • Many commercial products are available: Ethernet cards, GbE cards, memory modules, CPUs, etc.

  14. PMC Size CPU • RadiSys EPC-6315 • Equipped with Intel PentiumIII 800 MHz. • RedHat Linux 7.3 runs. • 32-bit 33/66 MHz PCI bus interface. You can login this CPU like your PC Even physics-analysis codes run on it! Contains Linux image

  15. ADC/TDC Module ADC/TDC Module ADC/TDC Module ADC/TDC Module Whole Design of the DAQ Platform VME 9U sized board mezzanine(add-on) module FIFO PMC PCI bus local bus Detector signals PMC CPU Network IF Bridge Trigger Module

  16. The COPPER-II This is our readout platform for the Super B factory.

  17. The COPPER-II VME-9U sized board PMC Processor ADC/TDC • 4 ADC/TDC slots • 1 MB × 4 FIFO • 32-bit local bus • 3 PMC slots • 32-bit 33MHz PCI bus • Local-PCI bridge • VME I/F ADC/TDC ADC/TDC Trigger ADC/TDC Generic PMC slot On-board Ether COmmon Pipelined Platform for Electronics Readout

  18. ADC/TDC Module Local bus I/F(for control) Photo of prototype card Digitizedsignal output 32 76×180 mm2 Trigger busy Clock, L1 trigger Clock = 43MHz, Reduced clock = depends on sub-system

  19. Readout Scheme How it works?

  20. Readout Scheme (1) • L1 Trigger comes… • Trigger module distributes trigger signal to the ADC/TDC modules. • Digitized data are held on the ADC/TDC module. • Data transfer to the COPPER-II starts. • COPPER-II FIFOs are filled… • The COPPER-II’s FPGA formats data (header and footer). • The COPPER-II’s FPGA counts each event size in FIFOs. • A PCI interrupt is generated when a total data size in the FIFOs exceed threshold to initiate the DMA.

  21. Readout Scheme (2) • PCI interrupt is generated… • The CPU checks the size of stored data counted by the COPPER-II. • The CPU starts data transfer from the FIFOs to CPU’s main memory using DMA. • A user process on the CPU is woke up after DMA… • The user process reads data and performs the data size reduction. • The data are transferred to the event builder…

  22. COPPER-II Performance Up to what input rate does COPPER-II work?

  23. Study Setup The COPPER-II ADC proto. Self trigger. Trigger rate is variable. CPU ADC proto. Trigger ADC proto. RX Vacant ADC proto. Ether Generates virtual ‘ADC’ data. Data size / event is variable.

  24. Maximum Trigger Rate Belle’s TDC data 16 bytes / ch. (drift chamber) 400 bytes/TDC-module corresponds to 100 ch. in one COPPER-II module. @ 416 bytes/ev/ADC-module Required trigger rate Typical trigger rate The COPPER-II works> 30 kHz input rate

  25. Profile of CPU Usage @ 416 bytes/ev/ADC-module • User time: ~2% • System time: ~20% • Idle time: ~78% = CPU power that is equivalent to P3 ~600MHz is still available • Large idle time fraction indicates the PCI bus works at the full performance. • 416 bytes / ADC-module / ev × 40 kHz × 4 ADC modules= 67 MB/s.

  26. Performance Degradation by Network Use @ 416 bytes/ev/ADC-module CPU user time: ~2% CPU system time: ~20% CPU idle time: ~78% Maximum acceptedtrigger rate: 40 kHz CPU proc.A FIFO Read CPU CPU user time: ~5% CPU system time: ~31% CPU idle time: ~64% Maximum acceptedtrigger rate: 32 kHz proc.A proc.B NetworkTransfer FIFO Read Ethernet 11MB/s RX Still works well.

  27. Schedule What are we planning?

  28. The COPPER-II prototype is being debugged. We start production of the COPPER-II from the begging of Feb.2004. We are planning to replace some Belle-DAQ parts with COPPER in 2005 summer. Schedule in Near Future

  29. Long Term Plan COPPER Proto. ADC/TDC Proto. We are here. Upgrade ASIC Development ADC Module Design TDC Module Design Performance Study Mass CPU Control User I/F Framework Event Building Farm L2/L3 Triggers Hardware Applications Software Applications Platform Integrated Test Goal

  30. Summary • We have developed new readout electronics, COPPER-II, for the Super B factory. • Design keywords are: Pipeline, Online data reduction, Flexibility, Compactness, and Commodity usage. • At the typical data size, the COPPER-II works up to 40 kHz trigger rate, which satisfies the requirements.

  31. Backup Slides Please help me…

  32. Introduction What is a readout electronics? What is Super B factory requiring for it?

  33. Introduction to Readout Electronics Detector TriggerSystem ReadoutElec. • Role • Signal digitization • Event buffering • Data suppression • Data transfer to EB EventBuilder Storage DAQ

  34. Keywords in the System Design • Five keywords, again… pipeline flexibility online data reduction compactness commodity use

  35. ADC/TDC Module All Components on a 9U-Size Board compactness Bridge CPU Network

  36. ADC/TDC Module trigger Trigger Module busy FINESSE TTRX Level 1 Trigger TRIGGER from the trigger module. BUSY response by ADC/TDC module. Data hold. Data-transfer to FIFO.

  37. ADC/TDC Module ADC/TDC Module  Readout FIFO • Data transfer to readout FIFO • 32-bit FIFO per ADC/TDC module. • Data size of each event is counted by FPGAand is stored to FIFO-word-counter FIFO. 32-bit data Readout FIFO (1 MB) FIFO WENA To CPU FIFO WCLK spied FPGA FIFO-word- counter FIFO-word-counter FIFO

  38. Readout FIFO  CPU: DMA PCI bus Readout FIFO filled Local-PCI Bridge CPU PCI interrupt Local bus Check event size PLX-9054 RadiSys EPC-6315 Initiate DMA Readout FIFO DMA data transfer DMA over PCI interrupt FIFO-word-counter FIFO

  39. COPPER Device Driver PCI interrupt Readout FIFO ready (every event) interrupt_handler() Read all available data in FIFO (mostly one event) while( data_in_FIFO ){ event_size = read_event_length_FIFO(); DMA_size += event_size; }; start_DMA( DMA_src_addr, DMA_dst_addr, DMA_size ); do_DMA() Return to user code. DMAbuffer PCI interrupt DMA over interrupt_handler() Read new data (if exist) after last do_DMA(). do_DMA() user_read() Return to user code.

  40. CPU  Event Builder PCI bus EB Ethernet Network • Are event-header/event-footer correct? • Are data contents consistent with pre-defined ones? • Generator module’s event counters = event tag from trigger module? • Are all event counters from 4 generator modules consistent? • Do event counters increase by 1 correctly? CPU serial line for debug

  41. Maximum Trigger Rate vs. Data Size Typical data size

  42. Data Transfer Speed on PCI vs. Data Size Typical data size

More Related