1 / 29

BESIII DAQ System BESIII Review Meeting IHEP · Beijing · China Sep. 16 - 17, 2002

BESIII DAQ System BESIII Review Meeting IHEP · Beijing · China Sep. 16 - 17, 2002. BESIII DAQ Scheme. Readout Crate System Controller – MVME2431-1. PowerPC 750™ 32-bit microprocessor 32MB of on-board ECC SDRAM Up to 1MB capacity for on-board firmware 8MB on-board Flash memory

avedis
Download Presentation

BESIII DAQ System BESIII Review Meeting IHEP · Beijing · China Sep. 16 - 17, 2002

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BESIII DAQ SystemBESIII Review MeetingIHEP · Beijing · China Sep. 16 - 17, 2002 BESIII Review Meeting

  2. BESIII DAQ Scheme BESIII Review Meeting

  3. Readout Crate System Controller – MVME2431-1 • PowerPC 750™ 32-bit microprocessor • 32MB of on-board ECC SDRAM • Up to 1MB capacity for on-board firmware • 8MB on-board Flash memory • One asynchronous serial debug port • 10/100Mbps Ethernet interface • 4-level requester, 7-level interrupter, and 7-level interrupt handler for VMEbus BESIII Review Meeting

  4. 3 Levels of Event Builder BESIII Review Meeting

  5. BESIII DAQ Tasks • Event Data Readout from FEE • Event Building • Event Filtering • Event Recording to Persistent Media • Run Control System • Monitoring (event, histogram display ...) • Message Reporting BESIII Review Meeting

  6. Data Rate Estimation • Level-1 Trigger Rate: 4KHz (2KHz Good Events) • Event Size: 12KBytes • Data Rate to be Processed by DAQ: 48MByte/sec • To be Recorded on Disk Array/Tape: 36MByte/sec BESIII Review Meeting

  7. Technologies • Multi-level Buffering • Switch Network • Parallel Computing • Easily Upgrade and Port • Easily Scale • Reliable / Stable / Error Recovery BESIII Review Meeting

  8. Online Data Flow BESIII Review Meeting

  9. PowerPC/VxWorks Based Test Environment BESIII Review Meeting

  10. Test Results – Programmed I/O System Controller: MVME2431-1 (350 MHz PowerPC 750) Operating System: VxWorks 5.4 Writing from CPU DRAM into VMEbus 32-bit transfers: 400 ns/32-bit write, 10MB/s 16-bit transfers: 400 ns/16-bit write, 5MB/s Reading from VMEbus into CPU DRAM 32-bit transfers: 1,100 ns/32-bit read, 3.6MB/s 16-bit transfers: 1,100 ns/16-bit read, 1.8MB/s BESIII Review Meeting

  11. Note: • The above VME access speeds were Programmed I/O speeds. • The program was written in C language and run in the context of the VxWorks shell. • The tests were conducted on an idle system with no other competing processes nor any concurrent activity on the VMEbus. BESIII Review Meeting

  12. Question: Can we use Programmed Read to read out data from ADCs/TDCs modules? BESIII Review Meeting

  13. Estimation of One VME Readout Crate (1) Basic Assumptions: • 1 crate = 16 modules • 1 module = 32 channels • Trigger rate: 4000 Hz • When one L1 trigger comes, 10% channels have signals. • Each channel has its own Buffer containing 16-bit data to be read. • Each module has a Hit Map (32-bit) • Data readout is in A24/D16 mode and 1.1 microsec/16bit read BESIII Review Meeting

  14. Estimation of One VME Readout Crate (1) (con’t) For each module: Data Ready Flag: 1 word Trigger Number: 1 word Hit Map: 2 word Data: 32 × 10% = 3.2 words For each crate: 16 × ( 1 + 1 + 2 + 3.2 ) = 115.2 words When trigger rate is 4000 Hz, readout time will be: 4000 × 115.2 bytes × 1.1 microsec = 0.51 sec BESIII Review Meeting

  15. Estimation of One VME Readout Crate (2) Basic Assumptions: • 1 crate = 16 modules • 1 module = 32channels • Trigger rate: 4000 Hz • When one L1 trigger comes, 10% channels have signals. • Each module has a Global Buffer containing 16-bit channel number and 16-bit data to be read. • Data readout is in A24/D32 mode and 1.1 microsec/32bit read BESIII Review Meeting

  16. Trigger Number Channel Number Event Data … … Channel Number Event Data Tail Trigger Number Channel Number Event Data … … Channel Number Event Data Tail … … Global Buffer Data Format31 0 BESIII Review Meeting

  17. Estimation of One VME Readout Crate (2)(con’t) For each module: Data Ready Flag: 1 DWord Header: 1 DWord Data: 32 × 10% = 3.2 DWords Tail: 1 DWord For each crate: 16 × ( 1 + 1 + 3.2 + 1 ) = 99.2 DWords When trigger rate is 4000 Hz, readout time will be: 4000 × 99.2 bytes × 1.1 microsec = 0.44 sec BESIII Review Meeting

  18. So, the answer is: • The system controller does not have enough time to read out data from ADCs/TDCs by using Programmed Read. • The way to get faster reads is to use the DMA engine resident on the Universe VME <=> PCI bridge chip. It is designed to copy larger chunks of data from one bus to the other. BESIII Review Meeting

  19. MVME2431 DMA Controller • The Universe II has a DMA controller for high performance data transfer between the PCI bus and VMEbus. • Transfer size • Any Number of bytes from 1 byte to 16 Mbytes. • Transfer Data Width • D08 (EO) • D16 • D16BLT • D32 • D64 • D32BLT • D64BLT (MBLT) BESIII Review Meeting

  20. Readout Mode: Global Buffer + DMA • Block size must be counted somewhere on board. • To avoid too many DMA operations (DMA Interrupts), it’s better for each ADC/TDC board to buffer data of multiple triggers (e.g. 64, 128 triggers), and then interrupt the system controller to start a DMA operation. BESIII Review Meeting

  21. DMA Transfer Some typical values for DMA transfers: D32 Read: 20 -- 23 MB/s D32 Write: 22 -- 25 MB/s D64 Read: 40 -- 45 MB/s D64 Write: 45 -- 50 MB/s Note: This is the raw speed of the hardware without any software overhead. BESIII Review Meeting

  22. Main Tasks of Readout • DMA transfer • Calculation • Data unpacking & packing • Network transmission. BESIII Review Meeting

  23. R & D Prototype BESIII Review Meeting

  24. File Server • DELL PowerEdge 2500 Dual Intel Pentium III 1.4 GHz Processors with 512K Cache • Integrated 10/100 Ethernet Adapter • 1 GBytes SDRAM • SCSI RAID Controller • Six 18GB SCSI Hard Drive, 10K RPM • Intel Pro 1000 Fiber Network Adapter • Red Hat Linux 7.3 Kernel 2.4.18-3 smp BESIII Review Meeting

  25. Online Farm & Readout PC Nodes • DELL GX260 Intel Pentium 4 1.8 GHz Processor with 256K Cache • Integrated 10/100/1000 Ethernet Adapter • 512 MBytes SDRAM • 40GB EIDE 7200 RPM Hard Drive • Red Hat Linux 7.3 Kernel 2.4.18-3 BESIII Review Meeting

  26. Ethernet Switch • Catalyst 2948G-L3 • 48 Ports of 10/100-Mbps Fast Ethernet • 2 Ports of 1-Gbps Gigabit Ethernet BESIII Review Meeting

  27. Network Performance (Preliminary) BESIII Review Meeting

  28. More Studies on Prototype • Performance Studies • Network performance in parallel mode • Raid disk performance write/read speed vs. config (raid0/raid5) • Data flow control software • Online filter software • Distributed histogram processing • Single event display BESIII Review Meeting

  29. Thanks! BESIII Review Meeting

More Related