1 / 22

A scalable gigabit data acquisition system for calorimeters for linear collider

A scalable gigabit data acquisition system for calorimeters for linear collider. Grant ANR-2010-0429-01. GASTALDI Franck. On behalf of the electronic & software team. Introduction: ILC detectors. Method: Imaging calorimetry ~100 10 6 channels/detectors Issues: Integration

mattox
Download Presentation

A scalable gigabit data acquisition system for calorimeters for linear collider

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A scalable gigabit data acquisition system for calorimeters for linear collider Grant ANR-2010-0429-01 GASTALDI Franck On behalf of the electronic & software team TIPP 2014 @ Amsterdam

  2. Introduction: ILC detectors • Method: Imaging calorimetry ~100 106 channels/detectors • Issues: • Integration • Power consumption • Ideas: • Detectors prototypes • Power pulsing (1% duty cycle ~25µW/ch) allowed due to the beam structure (5 Hz spills) • Switched on during > ~1ms of ILC bunch train and data acquisition • Bias currents shut down between bunch trains • Data acquisition and control • A single cable for everything • Scalable architecture • Reliable protocols & simplicity TIPP 2014 @ Amsterdam

  3. Introduction: ‘generic’ DAQ • In most cases, detectors and associated readout systems are designed, tested and approved before DAQ effort is undertaken • Our idea for this project is to design as a ‘generic’, scalable, and a self contained system, build around commercial components where possible. • This DAQ is then configured towards multiple ‘use-cases’. ILC calorimetry might not be the only customer Remark : This work follows a R&D from Univ.College London, Manchester Univ and Cambridge Univ that continued at LLR-EcolePolytechnique / IN2P3-CNRS TIPP 2014 @ Amsterdam

  4. Calorimeter DAQ: overview Slabs DAQ2 PC DIFs Network card 50 Mb/s 50 Mb/s ×n layers 1 Gb/s GDCC DCC (optional) ⋮×8 50 Mb/s ×7 : Machineclock 50 Mb/s Clock& Control DCC (optional) ⋮×8 50 Mb/s GDCC Slabs = detector unit : detector with integrated front-end electronics and sensors DIFs: Detector InterFace, servicing the detector unit GDCC: Giga-Data-Concentrator-Card: Concentrates data, fanin/fanout for clock and control data CCC: Clock & Control card: Fanout of clock and fast controls DCC: Data concentrator Card: optionnal extra level of data concentration 50 Mb/s Digital (Config, Control, Data) Clock & Sync Optic GigE or copper Debug USB TIPP 2014 @ Amsterdam

  5. Calorimeter DAQ: Serial Link (cont’d) HDMI connectors between DIF-DCC-GDCC-CCC - Commercial standard for consumer electronics - High-bandwith connection at low cost 3 twisted pairs + 2 optional • Reference clock (50 MHz), fan-out from CCC • Data in (fast control, slow-control) • Data out (slow control, data readout) TIPP 2014 @ Amsterdam

  6. DAQ: The DIF card • The DIF concept is generic in firmware, running on detector specific hardware • Based on low cost FPGA • Compact (73mm x 50 mm) • Control up to 10K channels • Functionalities are simple • VFE chip management (power pulsing, SC, DAQ) with a common interface • Local storage of SC data (Flash Ram) Architecture of the DIF FPGA TIPP 2014 @ Amsterdam

  7. DAQ: The GDCC card • Format : VME 6U (chassis with only J1 connector used for power distribution) • Format shared in 2 part (1/3 – 2/3) • 1/3 is the mezzanine with the HDMI connections • Reliability of mezzanine by a specific Samtec connector (SEAM and SEAF series: 160 pins) • Until 28 differential signals and 19 single ended • 2/3 is the GDCC “heart” with the main functionalities • Based around a Xilinx Spartan XC6SLX75 + Marvell component • USB is used to an extra access to the GDCC (debug for example) VME USB RJ45 & sfp fiber Main part Mezzanine part CCC HDMI 7 x DIFs HDMI TIPP 2014 @ Amsterdam

  8. DAQ: the GDCC card (cont’d) Functionalities: • Aggregate data from many DIF links and send it to the PC over Gigabit Ethernet link • The PHY layers is made by a specific component MARVELL88E1111 • Signaling between the DIF and the GDCC is made by 5 differential LVDS pairs in HDMI cable • Extract packet from the PC and execute the command sent (R/W register, DIF configuration packet, fast command) • Encapsulate data from DIF in Ethernet frame and send them to the PC E T H E R N E T FPGA Main Interface (based on several FSM And few Xilinx reference design) T O D I F Gemac lithe Homemade (from xilinx reference design) Totally free DIFs Links (Protocol fsm ser-des 8b/10b) MARVELL component mclk CCC interface trig Single architecture of GDCC card TIPP 2014 @ Amsterdam

  9. DAQ: The GDCC card (cont’d) GDCC frame to the PC is based on standard Ethernet format GDCC Header 3 kinds of frame Fast-command with a special Ethernet type 0x809 (GDCC DIF) Control data with a special Ethernet type 0x810 (GDCC DIF) Read-out data with the Ethernet type 0x811 (DIF  GDCC) Content of the DIF structure DIF SOF DIF EOF Data CLK Example of sending data from GDCC to DIF Data are sampled on rising-edge of clock Register packet from DIF TIPP 2014 @ Amsterdam

  10. DAQ: GDCC improvement • Put in place an UDP interface • Simple and fast protocol • Easy to be implemented in hardware • Does not require big resources Ethernet frame structure with UDP Header Currently under the first tests and after 3 days of sending a command to the DIF to read some registers (~ 3.106 times), there is no error. Architecture of UDP bloc TIPP 2014 @ Amsterdam

  11. DAQ: The DCC card (optional) • VME format • VME only used for the card power supply • 1 HDMI connection for the GDCC • Until 8 connections for the DIFs • Identical data rate at the input and output (50 Mb/s) Advantage: This card can be connected or disconnected in DAQ chain without modification of behavior. Architecture of DCC TIPP 2014 @ Amsterdam

  12. Calicoes Software The Acquisition chain • Ecal dedicated software suite • Based on the Pyrame framework (LLR) • Based on XML language • Allow to prototype rapidly a on-line system • Multi-media distribution(files, sockets and shared memories) • Online event-building Acquisition chain: software architecture TIPP 2014 @ Amsterdam

  13. Calicoes Software The Control-Command • Highly modular and distributed • Control the Ecal electronics but also the peripheral devices (Power supply, pulse generator,…) • Provides a high level state machine for final user • Scripting language (Python) • Good stability Global control-command architecture TIPP 2014 @ Amsterdam

  14. The system: beam test This DAQ has been used on the SiW-Ecal technical prototype for two years It has been used successfully for 4 test beams at DESY Typical setup is : (~2.5K Channels) 10 layers of detection, 10 DIFs, 2 GDCC 250 GBytes of data have been generated This system has been validated for 10 Hz of spill frequency (ILC requirement is 5 Hz) Exemple of event display 1e- (5GeV) 5 W plates between layers S/N > 14 DAQ chassis SLAB structure TIPP 2014 @ Amsterdam

  15. Conclusion • The aim who was to develop a DAQ system generic in nature, using commercial components where possible has been in most part attained • The tests had shown the ability of the DAQ to take a lot of data (~250 GB) • During the last beam test, 120000 configurations have been injected in the system in three weeks. It remained stable during all this time. • Currently, we improve our system with the implantation of a UDP block on GDCC. Next step Connect the DAQ to a real calorimeter system • 16 ASICs per ASU (under test today), will be up to 160 ASICs per layer Perspectives for ECAL(ILC) With this actual configuration and for 100M channels ECAL for example the setup will be: 12500 DCC, 2000 GDCC and 200 PC For reducing the number of card, the main work must be done on front end modules for easiness of integration TIPP 2014 @ Amsterdam

  16. Thankyou for your attention TIPP 2014 @ Amsterdam

  17. Back up TIPP 2014 @ Amsterdam

  18. Time line 2004-2008 30 layers 4000 channels 4000..10000 channels/dm3 1500 channels/dm3 4000 channels/dm3 ~24 X0, 20 cm thick ~2500 m2 active detectors ~100M readout channels S/N ~ 7.5 S/N ~ 15 TIPP 2014 @ Amsterdam

  19. Slabdetails ASUS with 16 Asics (180 x 180 mm) 1 Si Wafer with 256 pixels of 5X5 mm2 and thickness of 325 µm 190 mm 180 mm 70 mm 360 mm Battery charger application AVX BestCap BZ01 Afterregulator Slaboverview TIPP 2014 @ Amsterdam

  20. DIF card Slow control and read-out • Sent from the DAQ/control PC as a raw Ethernet frame • Passed to/from the DIF via GDCC/DCC with the following structure (protocol) Exemple of a fast decodind command at the DIF level Internallydecoded frame (test pin) IDLE SOF header data EOF DIF input:Standardpacket DIF output: Here: read out of 13x16b status registers (Reshaped into GDCC frame) Exemple of a decoding frame at the DIF level TIPP 2014 @ Amsterdam

  21. GDCC: some plots Trigger = startspill Data from DIF DIF SOF 0xfcff = startspill Example of data readout 0xfdff = start chip data Example of readout packet spied by wireshark RJ ~23 ps DJ ~166ps eyewidth ~19.75ns Example of Result of eyes diagram and jitter on data readout TIPP 2014 @ Amsterdam

  22. CCC card • Supplied by University of Cambridge in 2009 • Synchronize all sub-systems upon pre-spill warning • Until 8 HDMI connection • Distribute asynchronous fast trigger and/or busy signals • Capable to run stand-alone for distribute clock (50 MHz) and spill from an external trigger TIPP 2014 @ Amsterdam

More Related