The Hierarchical CBM Network Structure and the CBMnet V2.0 Protocol

1. DPG – Frühjahrstagung Mainz 2012 HK 57.2 University of Heidelberg Computer Architecture Group Frank Lemke, Sven Schenk, Ulrich Brüning 23.03.2012 The Hierarchical CBM Network Structure and the CBMnetV2.0 Protocol

2. DPG – Frühjahrstagung Mainz 2012 Outline • FAIR at GSI • Motivation • CBMnet V2.0 Network Protocol • CBM Network Structure • ASIC adaptions • Conclusion & Outlook

3. FAIR at GSI Darmstadt Germany Science Vol 318, www.sciencemag.org, 2 November 2007 Facility for Antiproton and Ion Research (FAIR) will extend the existingGSI accelerator and synchrotron (until 2016) with • Accelerator Facility focuses on five beam properties: Highest beam intensities Brilliant beam quality Higher beam energies Highest beam power Parallel operation • Eight different Experiments andCollaborations http://www.fair-center.de DPG – Frühjahrstagung Mainz 2012

4. DPG – Frühjahrstagung Mainz 2012 Compressed Baryonic Matter “Design and implementation of a hierarchical DAQ network,”Deutsche Physikalische Gesellschaft e.V. Fruehjahrstagung, March 10, 2008 “Design and implementation of a hierarchical DAQ network,”Deutsche Physikalische Gesellschaft e.V. Fruehjahrstagung, March 10, 2008 One of the FAIR experiments and collaborations is Compressed Baryonic Matter (CBM): • Investigating the highly compressed nuclearmatter using nucleus-nucleus collisions • Self-triggered detector system • Providing eight different kinds of detectors http://www.gsi.de/forschung/fair_experiments/CBM/index.html

5. DPG – Frühjahrstagung Mainz 2012 Motivation Standard Protocols do not support all special demands within CBM data acquisition (DAQ) system and an optimized design for the DAQ can results in better performance considering: Highbandwidth Synchronization mechanisms Limited space for hardware Radiation tolerance Self-triggered frontend electronics Reusability of protocol modules for different hierarchical devices => no protocol conversions

6. Features ofCBMnet V2.0 • Communication over one optical link supporting Data Transport Messages (DTM), Detector Control Messages (DCM) and Deterministic Latency Messages (DLM) • Optimized data utilization about 91 % (about 73 % considering 8b/10b) • Easy to use Interface • Highly modular CBMnet structure • Retransmission for Control and Data Messages • Fast and efficient administration packets • System wide clock recovery and synchronization • Deterministic link latency feature for well defined DLMs • Additional routing schemes available • Different physical layer implementations available DPG – Frühjahrstagung Mainz 2012

7. CBMnet V2 interface • Interfaces for three CBM network traffic classes for Data, Control and Synchronization • CBM Protocol Module Interface encapsulates communication Layer 1- 4 (4 partly) • All protocol specific features like initialization, retransmission are not visible for user Core Modules • Interface optimized for easy and efficient usage DPG – Frühjahrstagung Mainz 2012

8. FPGA Implementations For prototyping three different FPGA boards were used and supported by the CBMnet technology: • Active Buffer Board (ABB) • Data Combiner Board (DCB) • Read-Out Controller (ROC) Implementation specifics: • No elastic buffers • Only one clock source in the DAQ, recovered clocks used for sending • FPGA specific configurations • V4: Barrel Shifter Position, reset • V5 + V6: no Barrel Shifter, Alignment check reset • Priority reset insertion for DLMs and optimized HW DPG – Frühjahrstagung Mainz 2012

9. CBMnet – Beam Time Readout FEB Front-End Boards (FEB) for different detectors FEB ROC ROC DCB FEB Computer ROC ROC FEB ABB Computer FEB ABB FEB Computer ROC ROC DCB FEB ROC ABB Computer ROC FEB ABB DCB FEB FEB ROC ROC DCB FEB ROC ROC Clock and Sync Master FEB FEB ABB FEB ROC ROC DCB FEB ROC ROC FEB DPG – Frühjahrstagung Mainz 2012

10. Supporting new FPGA Versions • A Spartan 6 based ROC 3 Board is currently designed within the CBM Collaboration Delivering a lot of new use cases for the ROC It can now also serve as DCB for prototyping Enabling the full features of the CBMnet • Also a Fles Interface Board (FLIB) is planned Optimizes the First Level Event Selection (FLES) connection May serve as ABB substitution for prototyping Both boards will be supported by the CBMnet DPG – Frühjahrstagung Mainz 2012

11. Spartan 6 – Evalboard Tests DPG – Frühjahrstagung Mainz 2012

12. Generic CBMnet ASIC Support for CBMnet build-in modules for ASICs enables usage of one protocol in the complete network Provided modules: • Register File (RF) • I2C support for RF access • CTRL decode for CBMnet ctrl messages (a first version) • ASIC proven SERDES in 65nm • CBMnet with Master/Slave support • Unbalanced links: 1 up-stream and up to 4 down-stream links • Shift register chain support for analog designed ASIC parts DPG – Frühjahrstagung Mainz 2012

13. GPRF SPRF Generic CBMnet ASIC Structure Analog LVDS Blocks CBMnet Protocol Modules Analog Delay Elements Data 1x, 2x or 4x CBMnet interface Standard Cell SerDes Analog/Digital Read-out HW CBMnet V2.0 DLM 8b/10b De-/Encoder SERDES Data LVDS CLK Data LVDS RF Control Control LVDS CBMnet Digital Front-end Block I2C/JTAG GPRF : General Purpose Register File SPRF : Special Purpose Register File DPG – Frühjahrstagung Mainz 2012

14. First ASIC with CBMnet V2.0 –SPADIC V1.0 Tapeout in November 2011 ASIC received in February 2012 HK 34.7 : SPADIC – Self triggered readout ASIC for the CBM transition radiation detectors Tim Armbruster, Peter Fischer, Michael Krieger, and Ivan Peric DPG – Frühjahrstagung Mainz 2012

15. Conclusion & Outlook • Build up prototyp beam time read-out chains supporting current detectors and front-end boards combined with new ones • Support beam time DAQ systems for next beam times in June and October 2012 • Improve and expand generic CBMnet ASIC modules portfolio Our goal is the design of ASICs and DAQ elements required for the final CBM DAQ system DPG – Frühjahrstagung Mainz 2012

16. Questions ? Thank you for your attention ! DPG – Frühjahrstagung Mainz 2012