Download
1 / 19
190 likes | 287 Views
DAQ for new TPC readout. Ervin D énes, Zoltán Fodor KFKI, Research Institute for Particle and Nuclear Physics. DDL architecture. Concentrator. Standard detector/DAQ interface. Source Interface Unit. Forward Channel (Raw data). Backward Channel (Pedestals, control).
E N D
DAQ for new TPC readout Ervin Dénes, Zoltán Fodor KFKI, Research Institute for Particle and Nuclear Physics TPC readoutupgade meeting, Budapest
DDL architecture Concentrator Standard detector/DAQ interface Source Interface Unit Forward Channel (Raw data) Backward Channel (Pedestals, control) • Detector Data Link (DDL) : • Source Interface Unit • Transmission media • Destination Interface Unit Destination Interface Unit DAQ Read-out Receiver Card (D-RORC) PCI Bus TPC readoutupgade meeting, Budapest
ALICE Detector Data Link TPC readoutupgade meeting, Budapest
Read-Out Receiver Cards (D-RORC) • PCI-X adapter for 2 DDL max 100 MHz • PCI master:autonomous DMA TPC readoutupgade meeting, Budapest
DDL Features Interface: • Full duplex 32-bit data path on the destination interface (DIU card) • Half duplex 32-bit data path on the source interface (SIU card) • Full duplex flow control (XON/XOFF) • Interface clock up to 66 MHz (easy integration with PCI 66) • 264 MB/s peak data rate, 240 MB/s sustained bandwidth (max.) Implementation: • Duplex LC optical link up to 300 m • 2x FC or 2x GbE physical layer components • Small Form Factor Pluggable (SFP) optical transceivers • Bit error rate < 10-12 • Robust error detection: very low undetected bit error rate < 10-40 • Automatic link synchronization and management • Radiation tolerant Source Interface Extras: • Stand-by support (low power consumption) • In-system reconfiguration / Remote system upgrade • Monitoring of the aging of laser diode of optical transceivers TPC readoutupgade meeting, Budapest
Readout System Performance • Motherboard with dual Xeon CPUs @ 2.4 GHz • Six PCI-X slots, 4 bus segments (3+1+1+1), 2 controllers • Linux OS • ALICE Data-Acquisition software (DATE) TPC readoutupgade meeting, Budapest
#4 PCI #6 #2 #3 PCI #5 #2 PCI #4 PCI #3 Controller #1 Segment #1 PCI #2 PCI #1 Performance: 6 D-RORCs • Testing the fully populated PC using data source internal to PCI interface • Interoperability test • Measure the maximal input bandwidth Aggregate Bandwidth [MB/s] Normalized Bandwidth [MB/s/Ch] TPC readoutupgade meeting, Budapest
DDL Software • All functions accessible as interactive commands or API • Script-based interpreter for sequence of operations: • Sending command to the FEE • Reading FEE status • printing the status • comparing the status • polling the status • Downloading data into the FEE from a file • Reading data from the FEE • writing data into a file • comparing data with data in a file • TPC configuration: < 0.3 s FERO DDL D-RORC LDC define pedestal_addr 0x1FFF define enable_pedestal 0x2C reset SIU write_command enable_pedestal write_block pedestal_addr pedestal.hex %x read_and_check_block pedestal_addr pedestal.hex %x TPC readoutupgade meeting, Budapest
RORC Conc. FEE FEE command FEE command acq BUS DDL Transactions Writing a command to the FEE (e.g. CLEAR) No reply from the FEE write_command <FEE command> TPC readoutupgade meeting, Budapest
FEE address RORC Conc. FEE At least 6 FEE clocks FEE status FEE address FEE status acq BUS BUS BUS DDL Transactions Reading a status from the FEE read_and_print <FEE address> “<format>” [<stream>] read_and_check <FEE address> <expected status> <mask> read_until <FEE address> <expected status> <mask> <s> TPC readoutupgade meeting, Budapest
RORC Conc. FEE BUS FEE address FEE address Data block length acq acq acq end transaction DDL Transactions Writing data to the FEE (e.g. pedestal) write_block <FEE address> <file> [<format>] TPC readoutupgade meeting, Budapest
SW RORC Conc. FEE FEE address FEE address Data end block block length acq acq BUS BUS BUS end transaction DDL Transactions Reading data from the FEE (e.g. read back pedestal) read_and_check_block <FEE address> <file> [<format>] TPC readoutupgade meeting, Budapest
SW RORC Conc. FEE end run start run Rdy to receive Data Data end of event end of event event length event length acq acq BUS BUS BUS end transaction DDL Transactions Data collection TPC readoutupgade meeting, Budapest
D-RORC PC memory bank The Free FIFO Firmware Free FIFO page address page address page address PC CPU Event building Allocation of free pages TPC readoutupgade meeting, Budapest
D-RORC PC memory bank Direct Memory Access DDL Firmware PC CPU No involvement TPC readoutupgade meeting, Budapest
D-RORC PC memory bank page status page status page status length length length The Ready FIFO DDL Firmware Ready FIFO PC CPU Event building Delivery of filled pages TPC readoutupgade meeting, Budapest
TOFs VTPC1 VTPC2 MTPCR MTPCL CAMAC NA61 DAQ Configuration DDL links PC RORC RORC RORC VME to PCI RORC ? Busy Gate & Clock to Motherboards Trigger system Motherboards and concentrator TPC readoutupgade meeting, Budapest
Timing Diagram Evt 2. Evt 3. Evt 1. Gate for event Read FEE to mem 1 Read FEE to mem 2 Readout of FEE Transfer evt 1. to PC Transfer evt 2.. Transfer to PC Busy Busy generally = sampling time + FEE readout time, if the data flow fluid Busy = 1, during the readout of the FEE or no free memory to read it TPC readoutupgade meeting, Budapest
DDL/PCI bus: 100 MHz, 64 bits 800 MB/s Memory bus: 2.4 GHz, 64 bits 19.2 GB/s Bandwidth Estimations • Requirement: • 100 Hz, size: 6 MB 600 MB/s • SATA speed: • 6 Gb/s, 750 MB/s ? TPC readoutupgade meeting, Budapest
