1 / 29

Towards the FAIR Timing System using White Rabbit s

Towards the FAIR Timing System using White Rabbit s. Towards FAIR White Rabbit Timing System. GSI Timing Team: Marcus Zweig, Stefan Rauch, Mathias Kreider, Cesar Prados, Wesley Terpstra, Ralph B ä r, Dietrich Beck former members: Tibor Fleck, Sergio Mauro

hali
Download Presentation

Towards the FAIR Timing System using White Rabbit s

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards the FAIR Timing System using White Rabbits Towards FAIR White Rabbit Timing System Dietrich Beck, EE(BEL)

  2. GSI Timing Team: Marcus Zweig, Stefan Rauch, Mathias Kreider, Cesar Prados, Wesley Terpstra, Ralph Bär, Dietrich Beck former members: Tibor Fleck, Sergio Mauro CERN Timing Team: Tomasz Włostowski, Javier Serrano, Maciej Lipinski, Evangelia Gousiou, Erik van der Bij, Jean-Claude Bau, Pablo Alvarez GSI/EE: Jan Hoffmann, Nikolaus Kurz, Holger Brand, … Acknowledgements Dietrich Beck, EE(BEL)

  3. Towards FAIR… Dietrich Beck, EE(BEL)

  4. 23 November 2011 Dietrich Beck, EE(BEL)

  5. 8 December 2011 Dietrich Beck, EE(BEL)

  6. Bild von heute 12 January 2012 Dietrich Beck, EE(BEL)

  7. 12 January 2012 Dietrich Beck, EE(BEL)

  8. General Machine Timing System @ FAIR • parallel execution of beam production chains • cycles: 20ms to hours • trigger and sync. equipment actions • 1 µs precision in 99% of all cases • few ns precision for kickers • (ps for rf-systems: BuTiS) • many rings • > 2000 devices connected to timing system • large distances • robustness: lose at most one message per year Dietrich Beck, EE(BEL)

  9. Not: 1km distance:  5s propagation time 1s precision: requires compensation for cable length  Idea: Timing System Based on Time Equipment 1 Timing Master do this do this Equipment 2 Dietrich Beck, EE(BEL)

  10. Instead: equipment pre-programmed for autonomous action at a given time pre-programming via timing-events required: clock synchronization ns required distribution of information and execution of action are decoupled !!! timing-events must be sent “early enough” upper bound latency for transmission (e.g. 100 s): real-time! lossless transmission: robustness! Idea: Timing System Based on Time "Oh dear! Oh dear! I shall be too late!" ??? do this @ 15:03 Equipment 1 Timing Master do this @ 15:03 do that @ 15:01 Equipment 2 Dietrich Beck, EE(BEL)

  11. network: Gigabit-Ethernet PTP (Precision Time Protocol) IEEE1588-2008 • with value t1.. t4 one can • calculate one-way link delay • syntonize clock rate by tracking the value • calculate clock offset White Rabbit Cooking Recipe I: Clocks Dietrich Beck, EE(BEL)

  12. network: Gigabit-Ethernet PTP (Precision Time Protocol) IEEE1588-2008 free-running oscillators on network nodes need to re-sync often: lot’s of traffic, bad for determinism only 1-100 s synchronization of clocks SyncE (Synchronous Ethernet) receiver’s clock recovered from 125MHz carrier 8ns precision precise phase measurement and adjustment clock synchronization with sub-ns precision and low-ps jitter White Rabbit Cooking Recipe I: Clocks Dietrich Beck, EE(BEL)

  13. Clock propagation: Determinism “timing events shall arrive early enough” upper bound latency from timing master to nodes no handshake: use UDP or raw Ethernet (no TCP) switches implement cut-through for high-priority messages Robustness bit errors: Hamming code packet loss: Reed-Solomon code Redundancy redundant links between switches link aggregation (in combination with Reed-Solomon) multi-layered WR Switches System Timing Master WR Nodes GPS White Rabbit Cooking Recipe II: Network Dietrich Beck, EE(BEL)

  14. path asymmetry (two cables) impacts phase compensation White Rabbit PTP one bidirectional single-mode optical fiber different wave lengths for TX/RX: speed differs slightly (dispersion) link-delay-model relative delay coefficient α depends only on the type of fiber (off-line) fixed delays are measured on-line White Rabbit node VCO 125MHz VCO 20MHz FPGA WR clock!!! transceiver WR Cooking Recipe III: Implementation… host bus interface Dietrich Beck, EE(BEL)

  15. within FPGA: Wishbone “System-on-Chip” bus for interconnection of IP cores Inside the FPGA: FPGA Dietrich Beck, EE(BEL)

  16. idea: Connect distant Wishbone buses via a network protocol layer. Ethernet + Wishbone = Etherbone direct memory access to devices via White Rabbit network used for timing-events Transmission of Timing-Events: Etherbone Dietrich Beck, EE(BEL)

  17. development in the frame of CERN’s renovation and the FAIR projects collaboration: CERN, GSI, …, industry partners Open HardWare Repository: www.ohwr.org Open Source: Software and HDL “CERN Open Hardware License”: Hardware commercial production and support “The White Rabbit (WR) project is a multi-laboratory and multi-company effort to bring together the best of the data transfer and timing worlds …” (Proc. ICALEPCS 2009) The White Rabbit Project Dietrich Beck, EE(BEL)

  18. Timing Systems @ FAIR • Bunch phase Timing System (BuTiS) • invest € 15k per receiver station • ps precision (100ps/km accuracy) • distribution of • clocks!!! • NO time-stamps at FAIR • NO timing-events • HF group • one global BuTiS center • Talk P. Moritz, FAIR Technikforum • General Machine Timing System • based on White Rabbit • fairly cheap: € 600,- SPEC board • sub-ns synchronization • distribution of • (clock) • time-stamps • timing-events • BEL group • one global timing master • clock derived from BuTiS • Talk T. Fleck, FAIR Technikforum Remark: If you are interested time-stamps, you need White Rabbit. If you are interested in the most accurate clocks, you need BuTiS. If you are interested in both, you need both. Dietrich Beck, EE(BEL)

  19. The FAIR Control System (slightly simplified ) 1a: Data supply:set-values 1b: Data supply:schedules, alternatives • settings management (via normal network) • preloads set-values to the FECs (set-value can also be a ramp…) • provides schedule to timing-master • according to schedule, timing-master broadcasts timing-events to FECs (timing-events: time of execution and reference to a set-value) • timing-receiver of FEC: timely triggering of FEC, FEC applies set-value 2: broadcast timing-events 3. timing receiver Dietrich Beck, EE(BEL)

  20. Analogy to performing a classical piece of music: composer: settings management conductor: timing master musicians: FECs move of conductor's baton: timing-events - one timing-event may trigger multiple FECs The FAIR Control System (slightly simplified ) Data supply:set values Data supply:schedules, alternatives timing receiver Dietrich Beck, EE(BEL)

  21. Control System (application layer) API API API LSA (settings management) Timing Master BuTiS Post Mortem System Beam Interlock System Common Systems Alarm, Archive, Logging, Oscilloscope another link FESA BuTiS WR a link Equipment 1 Equipment 2 Equipment 3 Equipment 4 Equipment .. Equipment N Disclaimer: This is my personal view and may be incorrect and not up-to-date Dietrich Beck, EE(BEL)

  22. BuTiS and WR Dietrich Beck, EE(BEL)

  23. campus wide distribution system for timing-events time-stamps other machine or beam related telegrams not limited to accelerator controls available to “everyone” typically, data is propagated top-down other traffic only in exceptional cases kicker control bunch-to-bucket transfer beam interlock notification What the Timing System is About Dietrich Beck, EE(BEL)

  24. general purpose communication transmission of bulk data personnel protection system things not related to machine timing Exceptions only in well justified cases FECs not connected to FESA Info-Service part of machine protection system What the Timing System is NOT About Dietrich Beck, EE(BEL)

  25. @GSI: Set-up a first network 2012 usage in a field environment investigation of stability and long-term effects connection between new and existing timing system first tests with existing machines at GSI Next Steps… Dietrich Beck, EE(BEL)

  26. @GSI/CERN: WR starter kit (timing system demo)spring 2012 @Saclay: Low energy part of p-linac: first WR based timing system provided by GSI (early 2013) @CERN: Installation of WR based timing system at AD ring (2013) Installation of WR link between CERN and Gran Sasso (2012) …Next Steps Dietrich Beck, EE(BEL)

  27. Slovenia: In-Kind contribution timing receivers. Receivers and FPGA building blocks for FAIR timing system specified. Germany/GSI: In-Kind contribution General Machine Timing System: specification until March 2012. White Rabbit PTP link between two nodes established sub-ns synchronization with a jitter of hundred ps propagation of WR PTP over three layers of switches FPGA building blocks for simple timing system exists integration work in progress planning of WR network cabling in progress First WR nodes developed by EE Status Dietrich Beck, EE(BEL)

  28. www.ohwr.org https://www-acc.gsi.de/wiki/Timing/TimingSystemDocumentation (search for “FAIR timing system”) Further reading Dietrich Beck, EE(BEL)

  29. Moersbacher grund view towards SIS 100, 23 November 2011 Dietrich Beck, EE(BEL)

More Related