1 / 16

DPG – Frühjahrstagung Münster 2011 HK 20.7 University of Heidelberg Computer Architecture Group

DPG – Frühjahrstagung Münster 2011 HK 20.7 University of Heidelberg Computer Architecture Group Frank Lemke, Sven Schenk, Ulrich Brüning 22.03.2011. Experiences and results using the CBMnet protocol including precise time synchronization and clock distribution. Outline. Motivation

bert
Download Presentation

DPG – Frühjahrstagung Münster 2011 HK 20.7 University of Heidelberg Computer Architecture Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DPG – Frühjahrstagung Münster 2011 HK 20.7 University of Heidelberg Computer Architecture Group Frank Lemke, Sven Schenk, Ulrich Brüning 22.03.2011 Experiences and results using the CBMnet protocol including precise time synchronization and clock distribution

  2. Outline • Motivation • CBM Protocol Structure • CBM Network Synchronization • Beamtime results Dec. 2011 • Adaptions heading towards CBMnet V2.0 • Conclusion & Outlook

  3. Motivation Standard Protocols abilities are only partly reasonable for the special demands within CBM DAQ System • An optimized design of a CBM protocol will result in clearly better performance CBM unique requirements considered under the topics of • Space and cost limitations • High bandwidth • Specific system synchronization • Reusability of protocol modules for different Hierarchical Structure and Implementation

  4. CBM Network - Traffic Classes • Data Transport Messages (DTM) • Data Messages with CRC • Only with error detection • Detector Control Messages (DCM) • Control Messages with CRC • Retransmission on error • Deterministic Latency Messages (DLM) • 1-Bit Error correction • Additional Administration Packets • 1-Bit Error correction • IDLE, INIT, ACK, NACK … Arbiter Control Retransmission CRC Data CRC Link Link_init DLM ACK/NACK

  5. 16 Bit 16 Bit 16 Bit 16 Bit 16 Bit … … INIT … ACK … IDLE INIT IDLE 16 Bit 16 Bit 16 Bit … … IDLE DLM IDLE CBMnet V1.0 - Packet Structures DTM: DCM: DLM: Administrative packets:

  6. CBMnet - Communication Flow ComputingCluster Detector Frontend FEE Hub FEE Data flow DPB FEE FEE Hub FEE E/O FEE … Low Bandwidth High Bandwidth … Control flow ECS

  7. DLM SOP DATA CRC EOP DLM & Synchronization Specifics System requirements • Clock recovery and distribution for all system levels • Guaranteed deterministic timing during runtime at all times Special features provided by DLM usage • Heterogenous synchronized system initialization • Enables periodic time synchronization for epoch markers • Various special user defined event signalling with deterministic latency Implementation specifics • Priority request insertion • Deterministic and structural separated HDL coding • Specific HW requirements - FPGA configuration scheme and settings

  8. CBMnet - Synchronization Flow ComputingCluster Detector Frontend FEE Hub FEE DPB FEE FEE Hub FEE E/O FEE … … Periodic Synchronization Resynchronization ECS Initial HeterogenousSynchronization

  9. Cosy Beamtime December 2010 3 meter cables toABB and ECS Jitter cleaner device 50 meter cable toread-out controller DCB

  10. CBMnet Beamtime Results • Detector Control Messages • Worked without problems during complete beamtime tests • Data Transfer Messages • Merging of 4 data-streams within DCB into a single stream • No fatal data transmission error • No data flow problems • Deterministic Latency Messages • All counters values were reset synchronous • Synchronization using DLMs showed no problems

  11. Packet Format CBMnet V2.0 • Long messages extension (LME) requires a small message definition • Insertion of extension character • Reduction of buffer space • Balanced network message flow • LME packet well defined • No additional overhead for small messages • Type field within LME code required • Additional overhead circa 2% • Total overhead 11%

  12. Long Message Extension (LME) • Type: 4’b Identifier for LME, 1111 reserved • Message TAG (MTAG): 32 different long messages can be handled • Decrementing Sequence Number (DSN): Shows the amount of packets following • Maximal length = 256*64Byte = 16KByte

  13. … … … LME - Data Flow Influences • DLMs still easily insertable with priority request insertion into message streams without increasing complexity • DCMs and administration chars can be arbitrated inbetween LME packets • Numerous message streams can be merged into one aggregate data stream => LME has no major influence on the network flow characteristics

  14. Features towards CBMnet 2.0

  15. Conclusion & Outlook • Beamtime results show reliability and usability of CBMnet V1.0  • Planned extensions fit into CBMnet scheme for V2.0 • Interface stays almost the same • Implementation complexity should be manageable • The next step will be to include CBMnet logic into frontend ASICs • Also next generation FPGA ROCs and a possible read-out ASIC are planned and will probably be implemented soon

  16. Thank you for your attention ! Questions ?

More Related