1 / 14

FEDkit: a design reference for CMS data acquisition inputs

FEDkit: a design reference for CMS data acquisition inputs. V. Brigljevic a , G. Bruno a , E. Cano a , S. Cittolin a , S. Erhan b , D. Gigi a , F. Glege a , R. Gomez-Reino Garrido a , M. Gulmini a , J. Gutleber a , C. Jacobs a ,

ivana
Download Presentation

FEDkit: a design reference for CMS data acquisition inputs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FEDkit: a design reference for CMS data acquisition inputs V. Brigljevica, G. Brunoa, E. Canoa, S. Cittolina, S. Erhanb, D. Gigia, F. Glegea, R. Gomez-Reino Garridoa, M. Gulminia, J. Gutlebera, C. Jacobsa, M. Kozlovszkya, H. Larsena, I. Magrans De Abrila, F. Meijersa, E. Meschia, S. Murraya, A. Oha, L. Orsinia, L. Polleta, A. Racza, D. Samyna, P. Scharff-Hansena, C. Schwicka, P. Sphicasa,c, J. Varelaa aCERN, Geneva, SwitzerlandbUniversity of California, Los Angeles, USAcUniversity of Athens, Athens, Greece CERN/EP-CMD 9th Workshop on Electronics for LHC Experiments 1

  2. Topics • Introduction • S-LINK64 protocol and frame format • FEDkit • Hardware architecture • Software architecture • Environment • Performance • Next step for final CMS DAQ system: FRL CERN/EP-CMD

  3. CMS data acquisition (DAQ) • Several sub-detectors with various technologies • ~640 inputs to readout systems of DAQ • Throughput requirements: 200 MB/s average, bursts of 400MB/s • Common interface based on S-LINK64 specification CERN/EP-CMD

  4. S-LINK64 interface • Extension of already defined standard from CERN (S-LINK) • Recommends Common Mezzanine Card (CMC) format • Simple, FIFO-like interface from mother board to link daughter board • 64-bit wide, up to 100MHz • Flow control included CERN/EP-CMD

  5. 63 60 59 56 55 32 31 1 0 4 3 20 19 8 7 Hx $$ K BOE_1 Evt_ty LV1_id (24) BX_id (12) Source_id (10+2) FOV $$ K BOE_2 63 60 59 32 31 0 D Sub-detector payload D Sub-detector payload $$ K EOE_2 63 60 59 32 31 0 K EOE_1 xxxx Evt_lgth (24) CRC (16) xxxx Evt_stat(8) Tx $$ 63 60 59 56 55 32 31 16 15 12 11 4 3 0 Data frames • Encapsulation of the FED-specific data in common data format • Header contains identification of fragment, some statuses • Payload protected by CRC CERN/EP-CMD

  6. Generic III & Daughter boards • Generic III • PCI board with S-LINK64 connector embedded • Can be used as receiver (FEDkit) or sender (FED emulator) • Sender and receiver daughter boards communicate with LVDS cable (max 17m for 480MB/s cable speed) • Sender implements the S-LINK64 protocol for FED • Receiver board plugs into GIII CERN/EP-CMD

  7. Hardware architecture • Sender board • FPGA implements S-LINK64 protocol and controls LVDS chips • Receiver daughter board • LVDS chips and 32kB FIFO per link • GIII board • FPGA used to control reception FIFO and PCI 64-bit/66 MHz • Implements protocol with host PC • Hardware CRC check on the fly • Hardware link test mode CERN/EP-CMD

  8. Software architecture • Buffer loaning scheme • Parallel operation of DMA, block handling and fragment arrival notification CERN/EP-CMD

  9. Software architecture (cont.) • Software provided • Linux driver • FEDkit library • Example applications, application for XDAQ, the data acquisition framework used in CMS DAQ • OS bypass for performance • FEDkit library API (in C) matches high level functionality (simple to use) • Fragment handling functions provided CERN/EP-CMD

  10. FEDkit environment • FEDkit requires a PC with PCI 64bit 3.3V bus • Only OS supported is GNU/Linux • Can read FED data up to LVDS wire speed (480 MB/s) • Tested in PentiumIII, Xeon, Athlon machines • Fragment sizes up to 16MB • Hardware verification of CRC in receiver • Provided to FED developers • Successfully used in test beams (with FED emulator as source) CERN/EP-CMD

  11. Performance • Full LVDS wire speed always achievable • On Xeon more that twice the bandwidth of requirements • Software overhead hidden by DMA as fragments get bigger • Software overhead • on PIII: 5.3µs • On Xeon: 3.9µs • Hardware overhead (never masked) • 579 ns per fragment • 182 ns per block CERN/EP-CMD

  12. Final system design: FRL • FRL is the evolution of the FEDkit for the final system • Reuses a lot of FEDkit architecture • (Almost) same sender board, cable • Sends to intelligent NIC instead of host PC’s memory • Compact PCI form factor CERN/EP-CMD

  13. FRL architecture • LVDS link part very close to FEDkit’s one • Main FPGA uses similar protocol with intelligent network interface card (NIC) • Bridge FPGA allows control from host PC and spying (sampling of fragments) • Local DAQ through cPCI without full DAQ (but ~1kHz) for detector commissioning • Main FPGA used as data source for DAQ commissioning without FED data CERN/EP-CMD

  14. Conclusion • FEDkit implements the CMS DAQ readout functionality for lab tests • Realistic performance • Simple, direct access to fragments with provided library • Plug and play replacement in the final DAQ system with FRL http://cmsdoc.cern.ch/cms/TRIDAS/html/Documents.html CERN/EP-CMD

More Related