240 likes | 425 Views
Synchronous Device Interface at NSLS-II. Yuke Tian Control Group, NSLS-II, BNL (May 1, 2009 EPICS Collaboration Meeting, Vancouver). Outline. Synchronous, deterministic and reliable data distribution 2. Synchronous Device Interface 3. Synchronous Device Interface Extension MicroTCA
E N D
Synchronous Device Interface at NSLS-II Yuke Tian Control Group, NSLS-II, BNL (May 1, 2009 EPICS Collaboration Meeting, Vancouver)
Outline • Synchronous, deterministic and reliable data distribution • 2. Synchronous Device Interface • 3. Synchronous Device Interface Extension • MicroTCA • Summary EPICS Collaboration Meeting, Vancouver 2009
Synchronous, deterministic and reliable data distribution NSLS-II Accelerator Complex Requests to control systems from accelerator physics Distribute data in accelerator complex with: ● Synchronism: Data acquisition and distribution systems run off a single master oscillator. ● Deterministic : Data distribution with a fixed delay. ● High reliability: Data distribution with redundancy, no single point of failure. EPICS Collaboration Meeting, Vancouver 2009
Synchronous, deterministic and reliable data distribution: synchronism • Goal: run all circuitry in the accelerator off a single master oscillator • Advantages Accelerator are driven by many subsystems (RF, magnet/PS, diagnostics etc). Global synchronism offers repeatable, stable behavior. For example, fast orbit feedback. Synchronous stimulus-response measurement (such as response matrix, tune measurement etc) are easy in a fully synchronous system. Global synchronism offers global timestamp. There is no added jitter from crude local clock. EPICS Collaboration Meeting, Vancouver 2009
Synchronous, deterministic and reliable data distribution: synchronism Clock Domains In Accelerators EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: challenges for FOFB EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: answers for NSLS-II FOFB EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: cell controller EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: synchronism • Synchronism in NSLS-II 125MHz clock (in sync with 500 RF master clock) and fiducia will be distributed. We already have a very fine, low jitter clock. We can synchronize all circuit with this RF clock. We can timestamp every event with 2ns resolution. All the system are talking with the same clock. We will push the synchronism into power supply control system. Then, all subsystem (BPMs, cell controller, PS control etc) will be fully synchronous with RF clock. This will provide a lot of benefits for accelerator physics. EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: deterministic Latency calculations: EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: deterministic GTP interface test: 1.25Gbps 8-bit datawidth EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: deterministic GTP interface test: 2.5Gbps 16-bit datawidth EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: reliability • Synchronous Device Interface reliability: There are two 2.5Gbps links running in opposite directions. One is the main link and the other is backup link. When a link (or a pair of links) between two cell controllers is broken, the backup link is used to keep connectivity between any pair of cell controllers in the ring. The system has localized the broken link and re-configured automatically to maintain connectivity using backup link. The BPM data distribution is abstracted from upper application levels, and therefore corrector setting calculations are not altered when the system falls into the fault recovery operation mode. The fault is identified by the system, and error flags are delivered to the operator level. EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: extension EPICS Collaboration Meeting, Vancouver 2009
Synchronous Device Interface: Extension to PS Control EPICS Collaboration Meeting, Vancouver 2009
MicroTCA Request from distributed control/computing system (looks familiar ?) In 1979, Motorola was developing their new Motorola 68000 CPU and one of their engineers, Jack Kister, decided to set about creating a standardized bus system for 68000-based systems, which he called VERSAbus. http://en.wikipedia.org/wiki/VMEbus EPICS Collaboration Meeting, Vancouver 2009
MicroTCA EPICS Collaboration Meeting, Vancouver 2009
IEEE1014 40 MB/s ANSI/VITA 1.1 80 MB/s ANSI/VITA 1.5 320+ MB/s VITA 46 VME and/or Switch Fabrics 3U & 6U VITA 41 3 to 30 GB/s VME with Switch Fabric on P0 MicroTCA VME…VME64…VME64x…VME2eSST…VXS… 1981 1994 1997 2003 2005 Ethernet…10BASE5…10BASE2…10BASE-T…100BASE-T…1000BASE-X…10GBASE-X… 1982 1983 1984 1990 1994 1998 2002 EPICS Collaboration Meeting, Vancouver 2009
MicroTCA How about a cleaner solution ? VME backplane evolution MicroTCA EPICS Collaboration Meeting, Vancouver 2009
MicroTCA Emerson OM5080 Kontron OM9140 Elma Blu!Smart Schroff MicroTCA crate EPICS Collaboration Meeting, Vancouver 2009
MicroTCA Compare of VME and MicroTCA • Challenges of MicroTCA for accelerator control community: • IO modules • Software development. EPICS Collaboration Meeting, Vancouver 2009
Summary • SDI status: LBNL group (Larry Doolittle group) is leading the SDI core design. The communication through fiber links are done. BNL group is designing the SDI extension at PS control system. Get BPM data into cell controller by using Libera grouping features is under development. We are evaluating MicroTCA IOC. It will be test on EPICS/Linux. Then we will try to migrate to EPICS/RTEMS if necessary. EPICS Collaboration Meeting, Vancouver 2009
Summary • SDI features: Use the same clock to synchronously distribute data around nodes; Two directions transmitter and receiver to make it redundant and single node/connection fail safe; Serial chain link requires minimum wiring; Protocol independent of carrier/link speed: Rocket IO 2.5Gbps fiber link; 100Mbps Ethernet link; • SDI will be an open source design. EPICS Collaboration Meeting, Vancouver 2009
Acknowledgement Larry Doolittle (LBNL) Bob Dalesio (BNL) Carlos Serrano (LBNL) Joseph Mead (BNL) Thank you ! EPICS Collaboration Meeting, Vancouver 2009